Skip to main content
added 593 characters in body
Source Link
Henrik Schumacher
  • 109.5k
  • 7
  • 186
  • 323

After using them for several years, I have to admit that many Export routines in Mathematica are either broken or way too slow (a factor of 10 to 25 when compared to a "C++" routine, e.g. for such a simple task as exporting an array).

It is very unfortunate. However, the following can export the data within 1.6 seconds while the OP's version took 15.985 seconds on my machine. It allows for specifying how many digits have to be exported.

colsep = "\t";
rowsep = "\n";
prec = 16;
SetAttributes[doubletostring, Listable];
doubletostring[x_] := Internal`DoubleToString[x, False, prec];
Export[
    "a.txt",
    StringJoin[
      Map[
        row \[Function] StringJoin[Riffle[doubletostring /@ row, colsep], rowsep],
        e]
    ]
    ]; // AbsoluteTiming // First

1.60592

This timing is from Mathematica 11.3 on macOS v10.13.4 (High Sierra).

A hand-written C++ routine linked to Mathematica needs about 0.6 seconds to export the matrix e, so in principle, there is still room for improvement.

Since Internal`DoubleToString is undocumented, I have no idea what its second argument does (it appears to me that it has no effect which would be pretty weird). As ilian told me in this post, the second argument controls whether NumberMark shall be printed (False suppresses it). The third argument seems to specify how many leading digits have to be printed.

Some further speedup can be obtained by using ParallelMap instead of Map. Once the parallel kernels are set up and started, the timing is pretty close to the (unparallelized) C++ timing on my Quad Core machine:

Export["a.txt",
    StringJoin[
     ParallelMap[
      row \[Function] 
       StringJoin[Riffle[doubletostring@row, colsep], rowsep],
      e
      ]
     ]
    ]; // AbsoluteTiming // First

0.875412

Edit:

It is always a bit dangerous to rely on undocumented functions because that can be subject to change. As Domen pointed out in a comment, Internal`DoubleToString does not exist anymore in newer versions of Mathematica. (I am using version 14.1, and it's gone indeed.) But there is a new Internal`MRealToString with the same functionality and syntax. I guess the developers just renamed the function to align with the fact that the word "double" is typically not used in the Mathematica universe. We rather call them "machine real numbers" or just "machine reals".

After using them for several years, I have to admit that many Export routines in Mathematica are either broken or way too slow (a factor of 10 to 25 when compared to a "C++" routine, e.g. for such a simple task as exporting an array).

It is very unfortunate. However, the following can export the data within 1.6 seconds while the OP's version took 15.985 seconds on my machine. It allows for specifying how many digits have to be exported.

colsep = "\t";
rowsep = "\n";
prec = 16;
SetAttributes[doubletostring, Listable];
doubletostring[x_] := Internal`DoubleToString[x, False, prec];
Export[
    "a.txt",
    StringJoin[
      Map[
        row \[Function] StringJoin[Riffle[doubletostring /@ row, colsep], rowsep],
        e]
    ]
    ]; // AbsoluteTiming // First

1.60592

This timing is from Mathematica 11.3 on macOS v10.13.4 (High Sierra).

A hand-written C++ routine linked to Mathematica needs about 0.6 seconds to export the matrix e, so in principle, there is still room for improvement.

Since Internal`DoubleToString is undocumented, I have no idea what its second argument does (it appears to me that it has no effect which would be pretty weird). As ilian told me in this post, the second argument controls whether NumberMark shall be printed (False suppresses it). The third argument seems to specify how many leading digits have to be printed.

Some further speedup can be obtained by using ParallelMap instead of Map. Once the parallel kernels are set up and started, the timing is pretty close to the (unparallelized) C++ timing on my Quad Core machine:

Export["a.txt",
    StringJoin[
     ParallelMap[
      row \[Function] 
       StringJoin[Riffle[doubletostring@row, colsep], rowsep],
      e
      ]
     ]
    ]; // AbsoluteTiming // First

0.875412

After using them for several years, I have to admit that many Export routines in Mathematica are either broken or way too slow (a factor of 10 to 25 when compared to a "C++" routine, e.g. for such a simple task as exporting an array).

It is very unfortunate. However, the following can export the data within 1.6 seconds while the OP's version took 15.985 seconds on my machine. It allows for specifying how many digits have to be exported.

colsep = "\t";
rowsep = "\n";
prec = 16;
SetAttributes[doubletostring, Listable];
doubletostring[x_] := Internal`DoubleToString[x, False, prec];
Export[
    "a.txt",
    StringJoin[
      Map[
        row \[Function] StringJoin[Riffle[doubletostring /@ row, colsep], rowsep],
        e]
    ]
    ]; // AbsoluteTiming // First

1.60592

This timing is from Mathematica 11.3 on macOS v10.13.4 (High Sierra).

A hand-written C++ routine linked to Mathematica needs about 0.6 seconds to export the matrix e, so in principle, there is still room for improvement.

Since Internal`DoubleToString is undocumented, I have no idea what its second argument does (it appears to me that it has no effect which would be pretty weird). As ilian told me in this post, the second argument controls whether NumberMark shall be printed (False suppresses it). The third argument seems to specify how many leading digits have to be printed.

Some further speedup can be obtained by using ParallelMap instead of Map. Once the parallel kernels are set up and started, the timing is pretty close to the (unparallelized) C++ timing on my Quad Core machine:

Export["a.txt",
    StringJoin[
     ParallelMap[
      row \[Function] 
       StringJoin[Riffle[doubletostring@row, colsep], rowsep],
      e
      ]
     ]
    ]; // AbsoluteTiming // First

0.875412

Edit:

It is always a bit dangerous to rely on undocumented functions because that can be subject to change. As Domen pointed out in a comment, Internal`DoubleToString does not exist anymore in newer versions of Mathematica. (I am using version 14.1, and it's gone indeed.) But there is a new Internal`MRealToString with the same functionality and syntax. I guess the developers just renamed the function to align with the fact that the word "double" is typically not used in the Mathematica universe. We rather call them "machine real numbers" or just "machine reals".

added 484 characters in body
Source Link
Henrik Schumacher
  • 109.5k
  • 7
  • 186
  • 323

After using them for several years, I have to admit that many Export routines in Mathematica are either broken or way too slow (a factor of 10 to 25 when compared to a "C++" routine, e.g. for such a simple task as exporting an array).

It is very unfortunate. However, the following can export the data within 1.6 seconds while the OP's version took 15.985 seconds on my machine. It allows for specifying how many digits have to be exported.

colsep = "\t";
rowsep = "\n";
prec = 16;
SetAttributes[doubletostring, Listable];
doubletostring[x_] := Internal`DoubleToString[x, False, prec];
Export[
    "a.txt",
    StringJoin[
      Map[
        row \[Function] StringJoin[Riffle[doubletostring /@ row, colsep], rowsep],
        e]
    ]
    ]; // AbsoluteTiming // First

1.60592

This timing is from Mathematica 11.3 on macOS v10.13.4 (High Sierra).

A hand-written C++ routine linked to Mathematica (unfortunately, I cannot export it so easily)A hand-written C++ routine linked to Mathematica needs about 0.6 seconds to export the matrix e, so in principle, there is still room for improvement.

Since Internal`DoubleToString is undocumented, I have no idea what its second argument does (it appears to me that it has no effect which would be pretty weird). As ilian told me in this post, the second argument controls whether NumberMark shall be printed (False suppresses it). The third argument seems to specify how many leading digits have to be printed.

Some further speedup can be obtained by using ParallelMap instead of Map. Once the parallel kernels are set up and started, the timing is pretty close to the (unparallelized) C++ timing on my Quad Core machine:

Export["a.txt",
    StringJoin[
     ParallelMap[
      row \[Function] 
       StringJoin[Riffle[doubletostring@row, colsep], rowsep],
      e
      ]
     ]
    ]; // AbsoluteTiming // First

0.875412

After using them for several years, I have to admit that many Export routines in Mathematica are either broken or way too slow (a factor of 10 to 25 when compared to a "C++" routine, e.g. for such a simple task as exporting an array).

It is very unfortunate. However, the following can export the data within 1.6 seconds while the OP's version took 15.985 seconds on my machine. It allows for specifying how many digits have to be exported.

colsep = "\t";
rowsep = "\n";
prec = 16;
SetAttributes[doubletostring, Listable];
doubletostring[x_] := Internal`DoubleToString[x, False, prec];
Export[
    "a.txt",
    StringJoin[
      Map[
        row \[Function] StringJoin[Riffle[doubletostring /@ row, colsep], rowsep],
        e]
    ]
    ]; // AbsoluteTiming // First

1.60592

This timing is from Mathematica 11.3 on macOS v10.13.4 (High Sierra).

A hand-written C++ routine linked to Mathematica (unfortunately, I cannot export it so easily) needs about 0.6 seconds to export the matrix e, so in principle, there is still room for improvement.

Since Internal`DoubleToString is undocumented, I have no idea what its second argument does (it appears to me that it has no effect which would be pretty weird). As ilian told me in this post, the second argument controls whether NumberMark shall be printed (False suppresses it). The third argument seems to specify how many leading digits have to be printed.

After using them for several years, I have to admit that many Export routines in Mathematica are either broken or way too slow (a factor of 10 to 25 when compared to a "C++" routine, e.g. for such a simple task as exporting an array).

It is very unfortunate. However, the following can export the data within 1.6 seconds while the OP's version took 15.985 seconds on my machine. It allows for specifying how many digits have to be exported.

colsep = "\t";
rowsep = "\n";
prec = 16;
SetAttributes[doubletostring, Listable];
doubletostring[x_] := Internal`DoubleToString[x, False, prec];
Export[
    "a.txt",
    StringJoin[
      Map[
        row \[Function] StringJoin[Riffle[doubletostring /@ row, colsep], rowsep],
        e]
    ]
    ]; // AbsoluteTiming // First

1.60592

This timing is from Mathematica 11.3 on macOS v10.13.4 (High Sierra).

A hand-written C++ routine linked to Mathematica needs about 0.6 seconds to export the matrix e, so in principle, there is still room for improvement.

Since Internal`DoubleToString is undocumented, I have no idea what its second argument does (it appears to me that it has no effect which would be pretty weird). As ilian told me in this post, the second argument controls whether NumberMark shall be printed (False suppresses it). The third argument seems to specify how many leading digits have to be printed.

Some further speedup can be obtained by using ParallelMap instead of Map. Once the parallel kernels are set up and started, the timing is pretty close to the (unparallelized) C++ timing on my Quad Core machine:

Export["a.txt",
    StringJoin[
     ParallelMap[
      row \[Function] 
       StringJoin[Riffle[doubletostring@row, colsep], rowsep],
      e
      ]
     ]
    ]; // AbsoluteTiming // First

0.875412

Copy edited (e.g. ref. <https://en.wikipedia.org/wiki/MacOS_High_Sierra>).
Source Link

After using them for several years, I have to admit that many Export routines in Mathematica are either broken or way totoo slow (a factor of 10 to 25 when compared to a "C++" routine, e.g. for such asa simple task as exporting an array).

VeryIt is very unfortunate. However, the following can export the data within 1.6 seconds while the OP's version took 15.985 seconds on my machine. It allows for specifying how many digits have to be exported.

colsep = "\t";
rowsep = "\n";
prec = 16;
SetAttributes[doubletostring, Listable];
doubletostring[x_] := Internal`DoubleToString[x, False, prec];
Export[
    "a.txt",
    StringJoin[
      Map[
        row \[Function] StringJoin[Riffle[doubletostring /@ row, colsep], rowsep], 
        e]
    ]
    ]; // AbsoluteTiming // First

1.60592

This timing is from Mathematica 11.3 on macos 10macOS v10.13.4 (High Sierra).

A hand written-written C++ routine linked to Mathematica (unfortunately, I cannot export it so easily) needs about 0.6 seconds to export the matrix e, so in principle, there is still room for improvement.

Since Internal`DoubleToString is undocumented, I have no idea what its second argument does (it appears to me that it has no effect which would be pretty weird). As ilian told me in this post, the second argument controls wetherwhether NumberMark shall be printed (False suppresses it). The The third argument seems to specify how many leading digits have to be printed.

After using them for several years, I have to admit that many Export routines in Mathematica are either broken or way to slow (a factor of 10 to 25 when compared to a "C++" routine, e.g. for such as simple task as exporting an array).

Very unfortunate. However, the following can export the data within 1.6 seconds while the OP's version took 15.985 seconds on my machine. It allows for specifying how many digits have to be exported.

colsep = "\t";
rowsep = "\n";
prec = 16;
SetAttributes[doubletostring, Listable];
doubletostring[x_] := Internal`DoubleToString[x, False, prec];
Export[
    "a.txt",
    StringJoin[
      Map[
        row \[Function] StringJoin[Riffle[doubletostring /@ row, colsep], rowsep], 
        e]
    ]
    ]; // AbsoluteTiming // First

1.60592

This timing is from Mathematica 11.3 on macos 10.13.4.

A hand written C++ routine linked to Mathematica (unfortunately, I cannot export it so easily) needs about 0.6 seconds to export the matrix e, so in principle, there is still room for improvement.

Since Internal`DoubleToString is undocumented, I have no idea what its second argument does (it appears to me that it has no effect which would be pretty weird). As ilian told me in this post, the second argument controls wether NumberMark shall be printed (False suppresses it). The third argument seems to specify how many leading digits have to be printed.

After using them for several years, I have to admit that many Export routines in Mathematica are either broken or way too slow (a factor of 10 to 25 when compared to a "C++" routine, e.g. for such a simple task as exporting an array).

It is very unfortunate. However, the following can export the data within 1.6 seconds while the OP's version took 15.985 seconds on my machine. It allows for specifying how many digits have to be exported.

colsep = "\t";
rowsep = "\n";
prec = 16;
SetAttributes[doubletostring, Listable];
doubletostring[x_] := Internal`DoubleToString[x, False, prec];
Export[
    "a.txt",
    StringJoin[
      Map[
        row \[Function] StringJoin[Riffle[doubletostring /@ row, colsep], rowsep],
        e]
    ]
    ]; // AbsoluteTiming // First

1.60592

This timing is from Mathematica 11.3 on macOS v10.13.4 (High Sierra).

A hand-written C++ routine linked to Mathematica (unfortunately, I cannot export it so easily) needs about 0.6 seconds to export the matrix e, so in principle, there is still room for improvement.

Since Internal`DoubleToString is undocumented, I have no idea what its second argument does (it appears to me that it has no effect which would be pretty weird). As ilian told me in this post, the second argument controls whether NumberMark shall be printed (False suppresses it). The third argument seems to specify how many leading digits have to be printed.

added 58 characters in body
Source Link
Henrik Schumacher
  • 109.5k
  • 7
  • 186
  • 323
Loading
deleted 328 characters in body
Source Link
Henrik Schumacher
  • 109.5k
  • 7
  • 186
  • 323
Loading
deleted 328 characters in body
Source Link
Henrik Schumacher
  • 109.5k
  • 7
  • 186
  • 323
Loading
added 241 characters in body
Source Link
Henrik Schumacher
  • 109.5k
  • 7
  • 186
  • 323
Loading
added 63 characters in body
Source Link
Henrik Schumacher
  • 109.5k
  • 7
  • 186
  • 323
Loading
added 818 characters in body
Source Link
Henrik Schumacher
  • 109.5k
  • 7
  • 186
  • 323
Loading
edited body
Source Link
Henrik Schumacher
  • 109.5k
  • 7
  • 186
  • 323
Loading
Source Link
Henrik Schumacher
  • 109.5k
  • 7
  • 186
  • 323
Loading