3
$\begingroup$

Working with big files, I have to convert some numbers that are in string format into real MMA numbers. I know that I can use ToExpression but it's slow when compared to another forms.

For instance, for Integer case we can compare:

dataInt = ToString /@ RandomInteger[1000, {10^5}];
d1 = FromDigits /@ dataInt; // AbsoluteTiming
d2 = ToExpression /@ dataInt; // AbsoluteTiming

{0.050298, Null}
{0.475669, Null}

It's almost 10 times faster to use FromDigits. The question is, how can I make a equivalent for Real cases? Like:

dataReal = ToString /@ RandomReal[1000, {10^5}];
d1 = someFunction/@dataReal;//AbsoluteTiming
d2 = ToExpression/@dataReal;//AbsoluteTiming

I haven't found some Mathematica function to someFunction. I miss one way to force ToExpression to interpret the string in some specific way.

$\endgroup$
4
  • $\begingroup$ This begs the question as to why you aren't just reading the numbers as ... numbers, ReadList[] style. $\endgroup$
    – dwa
    Commented Feb 15, 2013 at 3:48
  • 1
    $\begingroup$ I have numbers with comma as separator like "23,23", so I have to use StringReplace and then convert into number. $\endgroup$
    – Murta
    Commented Feb 15, 2013 at 3:49
  • $\begingroup$ Can you read as import as "Table"? $\endgroup$
    – s0rce
    Commented Feb 15, 2013 at 3:49
  • $\begingroup$ Import is slow, I use ReadList that hasn't this option. $\endgroup$
    – Murta
    Commented Feb 15, 2013 at 3:53

2 Answers 2

9
$\begingroup$

I found in this answer the function Internal`StringToDouble that does exactly what I was looking for:

dataReal = ToString /@ RandomReal[1000, {10^5}];
d1 = Internal`StringToDouble /@ dataReal; // AbsoluteTiming
d2 = ToExpression /@ dataReal; // AbsoluteTiming
d1 == d2
{0.045391, Null}
{0.544008, Null}
True

As you can see, 10x faster then ToExpression.

As of 12.3, the Internal function is Internal`StringToMReal

$\endgroup$
1
$\begingroup$

Well yours is better and faster but I'll post this anyways. This is the best I can do for the moment. For 10^6 values its about 15% faster then ToExpression:

First@ImportString[
    StringReplace[StringDrop[StringDrop[ToString[dataReal], 1], -1], 
     "," -> ""], "Table"]; // AbsoluteTiming
$\endgroup$
1
  • $\begingroup$ tks @S0rce! I haven't sow ImportString, it has some Real options. $\endgroup$
    – Murta
    Commented Feb 15, 2013 at 4:01

Not the answer you're looking for? Browse other questions tagged or ask your own question.