3
$\begingroup$

I'm trying to implement an efficient TSV exporter based on the code from this answer. It works sufficiently quickly but the problem is that OutputForm shows about 6 most significant digits of a number, so I am forced to use DecimalForm to get all the digits as follows:

dat = RandomReal[{-100000, 100000}, {1000, 3000}];
PrependTo[dat, Table["header_" <> ToString[i], {i, 3000}]];

str = OpenWrite["speedtest.tsv"];
WriteString[str, DecimalForm[Row[#, "\t"], Infinity], "\n"] &~Scan~dat // AbsoluteTiming
Close[str];

But DecimalForm often drops the last digit of a number or changes it by 1 or rounds, here is an example (Mathematica 11.3):

DecimalForm[{-38741.90768741473, -7756.0224337393065, 
-39201.327646149446, -38666.237619983534, -7703.96642749069, 
-38539.13511093822, -38442.888322157516}, Infinity]
{-38741.90768741472, -7756.022433739306, -39201.32764614944,
 -38666.23761998353,-7703.966427490689,-38539.13511093823,-38442.88832215751}

One can see that the last digit of the first number is changed from 3 to 2, for the second, third and fourth numbers it is just dropped and so on... With Export everything works as expected but very slow:

ExportString[{-38741.90768741473, -7756.0224337393065, 
-39201.327646149446, -38666.237619983534, -7703.96642749069, 
-38539.13511093822, -38442.888322157516}, "TSV"]
"-38741.90768741473
-7756.0224337393065
-39201.327646149446
-38666.237619983534
-7703.96642749069
-38539.13511093822
-38442.888322157516
"

Is it possible to get the correct digits from DecimalForm? Or may be there is another efficient way to get in OutputForm all the digits for numbers? Note that my table also contains strings which should be exported without extra quotes.

$\endgroup$
  • $\begingroup$ This and this might be of interest for you. $\endgroup$ – Henrik Schumacher May 14 '18 at 18:53
  • $\begingroup$ Alexey, I just edited the C++ code. It works now without precision loss. Have a try. $\endgroup$ – Henrik Schumacher May 14 '18 at 19:41
  • 1
    $\begingroup$ @HenrikSchumacher Thank you for your efforts. The method from the first link produces much more correct output than the method from my question if we increase the precision to 17, but in this case it produces also redundant digits due to the binary representation. I've found that we can get the same result (but 20% faster) if we increase the precision of the original numbers to 17 before exporting with the method from my question. I'll try the C++ solution when I install the compiler. $\endgroup$ – Alexey Popkov May 14 '18 at 23:47
  • $\begingroup$ It looks like Export to "Table" merely applies ToString[#, InputForm]& to every number in the table for conversion into a string. So one workaround is the following change: WriteString[str,Row[InputForm[#,NumberMarks->False]&/@#,"\t"],"\n"]&~Scan~dat, but it is 10 times slower than the DecimalForm method (and 2 times faster than Export). $\endgroup$ – Alexey Popkov May 15 '18 at 0:41
  • $\begingroup$ I don't have DecimalForm in version 10.1. Do you find any use in converting ToString en masse and post processing? Something like StringReplace[ ToString[dat, InputForm, PageWidth -> ∞], {", " -> "\t", "{" -> "\n", "}" -> ""}] $\endgroup$ – Mr.Wizard May 24 '18 at 0:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.