How do you convert a string containing a number in C scientific notation to a Mathematica number?

Question

Suppose I have a string containing the C-representation of a floating point number; for example

s = "1.23e-5"

and I want to convert this to a Mathematica number. How can I do this?

ToExpression[s] gives Plus[-5, Times[1.23`, e]].

Answer 1

I think probably the cleanest way to do this (at least, if you have only a single string, or are faced with a separate string for each number you wish to convert as a result of some other process) is to use the undocumented function Internal`StringToMReal, i.e.:

Internal`StringToMReal["1.23e-5"]

which gives:

0.0000123

However, if you are trying to convert many such numbers at once, the standard, documented methods (Import, Read, etc.), are likely to represent better approaches.

CAVEAT!

Be aware, that Internal`StringToMReal almost always returns a Wolfram Real, even if the input is not a valid number. Also, in some cases it cannot interpret an input that is supposed to be a real number without preprocessing. Some of the examples:

{Internal`StringToMReal["0.23e-05"],
 Internal`StringToMReal["0.23e-0.5"]}

{Internal`StringToMReal["1.0000000000000000"],
 Internal`StringToMReal["1.00000000000000000"]}

{Internal`StringToMReal["-1"],
 Internal`StringToMReal[" -1"]} (* Can be avoided by StringTrim *)

Out[] := {-1., 1.}

{Internal`StringToMReal["1/2"],
 Internal`StringToMReal["1./2."]} (* Should use Internal`StringToMRational instead *)

{Internal`StringToMReal["."],
 Internal`StringToMReal["x"],
 Internal`StringToMReal["-"]}

Out[] := {0., 0., 0.}

It is alarming that Internal`StringToMReal["x"] silently returns 0.0 without any error. Of course some of these examples are arguably malformed, and can be easily prechecked. However, you cannot do that uniformly with StringMatchQ[..., NumberString], because e.g. StringMatchQ["1.23e-5", NumberString] returns False.

---

Before version 12.3 the proper way to invoke this was:

Internal`StringToDouble["1.23e-5"]

Answer 2

s = "1.23e-5"

# &[Read[#, Number], Close@#]&[ StringToStream@s ]

Which is not as good as what you started with. Note that it is important to close the stream.

---

Szabolcs says this is difficult to read. That was surely not my intention. You could also write it verbosely like this:

fromC =
    Module[{output, stream},
      stream = StringToStream[#];
      output = Read[stream, Number];
      Close[stream];
      output
    ] &;

fromC[s]

Answer 3

On version 7 Internal`StringToDouble fails on long strings, and fails to recognize exponents:

Internal`StringToDouble["3.1415926535897932385"]

Internal`StringToDouble /@ {"3.14159", "3.14159e-02", "3.14159e+02"}

$Failed["Bignum"]

{3.14159, 3.14159, 3.14159}

This sent me looking for another way to convert numeric strings. Using Trace on ImportString I found another internal function that does what I need: System`Convert`TableDump`ParseTable.

Being an internal function is it not error tolerant and if fed bad arguments it will crash the kernel. The syntax is as follows:

System`Convert`TableDump`ParseTable[
  table,
  {{pre, post}, {neg, pos}, dot},
  False
]

table  :   table of strings, depth = 2; need not be rectangular.  
pre    :   List of literal strings to ignore if preceding the digits (only first match tried).  
post   :   List of literal strings to ignore if following the digits (only first match tried).  
neg    :   literal string to interpret a negative sign (`-`).  
pos    :   literal string to interpret a positive sign (`+`).  
dot    :   literal string to interpret as decimal point.

(Using True in place of False causes a call to System`Convert`TableDump`TryDate that I do not yet understand.)

Example:

System`Convert`TableDump`ParseTable[
  {{"-£1,234.141592653589793e+007"}, {"0.97¢", "140e2kg"}},
  {{{"£"}, {"kg", "¢"}}, {"-", "+"}, "."},
  False
]

{{-1.2341415926535898*^10}, {0.97, 14000.}}

Answer 4

Another solution would be to use SemanticImportString (new in 10).

Borrowing some code from Mr.Wizard so that I can compare my solution to his:

strings =
  ToString @ Row[RandomChoice /@ {{"-", ""}, {#}, {"e"}, {"-", ""}, Range@12}] & /@ 
    RandomReal[{0, 10}, 15000];

Needs["GeneralUtilities`"]

Internal`StringToDouble /@ strings // AccurateTiming

System`Convert`TableDump`ParseTable[
  {strings}, {{{}, {}}, {"-", "+"}, "."}, False
] // AccurateTiming

Interpreter["Number"][strings]   // AccurateTiming

SemanticImportString[
     StringJoin[Riffle[strings, ";"]],
     {"Number"}, 
     "List",
     Delimiters -> ";"
] // AccurateTiming

0.00671892

0.00504799

12.980645

0.0426966

Now as you can see there is still an order of magnitude, but at least SemanticImport is strict with things that are not numbers, while Internal`StringToDouble["foo"] returns 0..

Some of the types in Interpreter will benefit from using SemanticImport internally when called on lists of strings in the future.

As far as the current speed of Interpreter there is only so much you can gain if you want to support things like

Interpreter[
    Restricted["Number", {0, 10, 0.5}],
    NumberPoint -> "baz",
    NumberSigns -> {"foo", "bar"}
]["bar5baz5"]

5.5

Answer 5

First[ImportString["1.23e-5", "List"]] might be slightly less hack-y than your suggestion in the comments...

Answer 6

Version 10 introduced Interpreter which would seem suited to this task:

Interpreter[form]
represents an interpreter object that can be applied to a string to try to interpret it as an object of the specified form.

Interpreter["Number"]["1.23e-5"]

0.0000123

Unfortunately it seems that like many new-in-10 functions this is far from optimized. In fact I would say its performance is nothing short of abysmal for this particular task.

Some string data to test with:

strings =
  ToString @ Row[RandomChoice /@ {{"-", ""}, {#}, {"e"}, {"-", ""}, Range@12}] & /@ 
    RandomReal[{0, 10}, 15000];

Timings for Interpreter against StringToDouble and ParseTable (see the other answers):

Needs["GeneralUtilities`"]

Internal`StringToDouble /@ strings // AccurateTiming

System`Convert`TableDump`ParseTable[
  {strings}, {{{}, {}}, {"-", "+"}, "."}, False
] // AccurateTiming

Interpreter["Number"] /@ strings   // AccurateTiming

0.0052075

0.00645107

10.625608

At more than three orders of magnitude slower than the old methods the new function is simply not appropriate for general use. Hopefully it will be improved in a future release.

Answer 7

updated based on comment feedback

One more approach, using LibraryLink. Create a C file called strto.cpp as follows:

#include <cstdlib>
#include "WolframLibrary.h"

EXTERN_C DLLEXPORT int wolfram_strtol(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
  char *string;
  mint base;
  mint result;
  string = MArgument_getUTF8String(Args[0]);
  base = MArgument_getInteger(Args[1]);
  result = strtol(string, NULL,base);
  MArgument_setInteger(Res,result);
  return LIBRARY_NO_ERROR;
}

EXTERN_C DLLEXPORT int wolfram_strtod(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument Res) {
  char *string;
  mint base;
  mreal result;
  string = MArgument_getUTF8String(Args[0]);
  result = strtod(string, NULL);
  MArgument_setReal(Res,result);
  return LIBRARY_NO_ERROR;
}

This is a very thin wrapper for the C++ strtol and strtod standard library functions.

Create the library:

Needs["CCompilerDriver`"];
lib = CreateLibrary[{"wolfram_strto.cpp"}, "wolfram_strto"]

Load the two library functions:

strtol = LibraryFunctionLoad[lib, "wolfram_strtol", {"UTF8String", Integer}, Integer];
strtod = LibraryFunctionLoad[lib, "wolfram_strtod", {"UTF8String"}, Real];

Test the basics:

strtol["104", 10]

This should return the integer 104

strtod["10e4"]

This should return the real 100000.

Check some harder cases:

strtod /@ {"3.14159", "3.14159e-02", "3.14159e+02", "1.23e-5", "1E6", "1.734E-003", "2.12e1"}

Try a hex number:

strtol["0x2AF3", 0]

This should return 10995 (e.g. same as 16^^2AF3)

Measure the elapsed time to 15,000 randomly generated reals:

strings = ToString @ Row[ RandomChoice /@ {{"-", ""}, {#}, {"e"}, {"-", ""}, Range@12}] & /@ RandomReal[{0, 10}, 15000]
First@AbsoluteTiming[ strtod /@ strings]

Returns in about 0.017 seconds on my machine.

For big numbers, there is another difference:

Internal`StringToDouble["1e4000"]
strtod["1e4000"]

The StringToDouble function gives $Failed["IEEE Exception"] and the strtod function gives DirectedInfinity[1].

In the case of underflow you get, respectively, $Failed["IEEE Underflow"] and 0.

Also, StringToDouble recognizes WL notation (e.g. 6.022*^23) and strtod does not recognize this format.

Answer 8

May be one can try the following

convert[inp_?StringQ] := ToExpression@StringReplace[inp, "e" -> "*10^"];

Answer 9

Here is a mathematica function which accepts a string and return a number or a string containing an error message.

ConvertScientificNumberStringToNumber[string_String] := Block[
   {regexSciNum, regexNumOnly, regexNumEOnly},
   regexSciNum = "^ *(\\+|-)?(\\d+(\\.\\d+)?|\\.\\d+)((e|E)((\\+|-)?\\d+)?)? *$";
   regexNumOnly = "^ *(\\+|-)?(\\d+(\\.\\d+)?|\\.\\d+) *$";
   regexNumEOnly = "^ *(\\+|-)?(\\d+(\\.\\d+)?|\\.\\d+)(e|E) *$";
   If[! StringMatchQ[string, RegularExpression[regexSciNum]],
     Return["String is not a valid Scientific Format Number"];
   ];
   If[ StringMatchQ[string, RegularExpression[regexNumOnly]],
     Return[ToExpression[string]];
   ];
   If[ StringMatchQ[string, RegularExpression[regexNumEOnly]],
     (* If nothing appears after e|E then We need to strip everything after e|E *)
     Return[ToExpression[StringReplace[string, RegularExpression["(e|E)(.+)?$"] -> ""]]]
   ,
     Return[ ToExpression[StringReplace[string, RegularExpression["(e|E)"] -> "*^"]]]
   ];
   Return["Error we should not reach this point in the function."];
];

Answer 10

This works for me with large data (1E6 points) in Ver 8.0.1:

test = Import["scope_29_1.csv", "Data"];
test2 = ToExpression[Drop[test, 2]];

"Data" forces mathematica to convert 1.734E-003 into 0.001734 but keeps as string because the first 2 lines contains names. "Drop" Keeps the first non-numerical lines out.

Answer 11

ToExpression@StringReplace[s, "e" -> "*10^"]