Alternative to Dump Save

Question

When dealing with large matrices I like to store them in the memory once for all, such that when I restart Mathematica I do not have to wait for a very long computational time of order of minutes.

To provide a simple example, my current solution is a "DumpSave"-ing local objects, e.g.

A=Table[i+j,{i,1,4},{j,1,4}];
DumpSave[LocalObject["A"],A];
Get[LocalObject[A]];

This saves the object persistently in the folder: ~/../.Wolfram/Objects.

However I do not quite like this, as I may overwrite some of the file by mistake and when switching from a PC to another I have to calculate the matrix again.

Ideally I would create another auxiliary notebook containing the full expression of the large matrix and just load it along with the main notebook, but most of time sis not possible to do that, as Mathematica is not able to fully expand such matrices.

Would you suggest a suitable alternative?

Thanks

Answer 1

For large matrices of numeric values, I suggest to export to the MATLAB MAT-format. This is a binary format for matrices (and tensors) only and thus, it is much faster than exporting arbitrary symbols.

file = "data.mat";
A = RandomReal[{-1, 1}, {1000, 1000}];
Export[file, A];
B = Import[file][[1]];
B == A

(* True *)

This writes the matrix to the file "data.mat" in the user's home directory. Of course, the value of file can be changed to any valid filename on your machine.

Answer 2

A simple and fast way to do this is to use Export with a .mx format file like "/path/file.mx"

A = RandomReal[{-1, 1}, {3000, 3000}];
Export["~/file.mx", A];

.

B = Import["~/file.mx"];

Timing

Here I compare ".mx" vs ".mat"

A = RandomReal[{-1, 1}, {3000, 3000}];
{
  AbsoluteTiming[Export["~/fileMX.mx", A]; ]
, AbsoluteTiming[Export["~fileMAT.mat", A]; ]
 }
 (*{{0.243336, Null}, {2.57043, Null}}*)

So at Exporting ".mx" is $10.56\times$ faster than ".mat"

{
   AbsoluteTiming[Import["~/fileMX.mx"]; ]
 , AbsoluteTiming[Import["~/fileMAT.mat"]; ]
 }
 (*{{0.033811, Null}, {0.320246, Null}}*)

at Importing ".mx" is $9.47\times$ faster than ".mat"

This is a strange result, as ".mat" is a matrix specialist (optimized for matrices) while ".mx" is a generalist.