How do I save/export a list so that it can later be easily imported as a list again?

Question

I have a couple of big lists (each of them contains elements that are themselves lists of two elements, i.e. an element of each list that I have is of the form {x, y}) that I want to save/export. Their generation namely takes hours, and I don't want to do this every single day.

I looked at the Mathematica help section on this (http://reference.wolfram.com/mathematica/tutorial/ImportingAndExportingData.html), but found that if I follow the example, I can't just import the data back into a list as it was before. It just ends up being something really messy.

So given such a list, say, list = {{1,2}, {1,3}, ... , {500, 500}}, what do I do, so that the next day I can just write list = Import[...]?

Answer 1

I'd use .mx files (Export / Import in "MX" format):

Export["myFile.mx",list]

and

Import["myFile.mx"]

This is fast, and does not really involve serialization / parsing in the usual sense (via strings). In other words, mx files bypass the high-level parsing, populating internal structures at lower level. In addition, mx files preserve packed arrays.

Answer 2

I provide two ways:

1) Human readable

data1 = RandomInteger[100, {25, 25}];
data2 = RandomReal[100, {25, 25}];
Save["humanReadable.m", {data1, data2}];

Unset[{data1, data2}]
Get["humanReadable.m"];
Dimensions@{data1, data2}

{2, 25, 25}

Note, that you can dump many different variables with ease, and the file is in easy to read Mathematica syntax allowing all kinds of symbolic and numeric data without any manual serialisation. By default Save appends which can be convenient, but ofc. must be kept in mind.

Saving the data from other sources in such format might be very handy, as you can use all of the Mathematica syntax including comments. For example I use it for measurement control software data dump.

2) Platform independent binary

Obviously the upside of binary format is smaller file size and loading time in case of big data.

In addition to mx data format already discussed, there is version and platform independend format wdx. Again there is a way to export and import it with symbol names attached.

DumpSave["platformIndependendBinary.wdx", {data1, data2}];
Unset[{data1, data2}]
Get["platformIndependendBinary.wdx"]

And if you don't want to fix / remember the variable names you can

DeleteFile["platformIndependendBinary.wdx"]
Export["platformIndependendBinary.wdx", {data1, data2}]
Unset[{data1, data2}]
{data1, data2} = Import["platformIndependendBinary.wdx"];

The same method works for .m files for human readable text format.

The only downside of wdx compared to mx is speed.

Answer 3

Probably the best way is to do

Export["mydata.txt", list, "Table"]

then later

Import["mydata.txt", "Table"]

Be sure to explicitly specify the data format: "Table". Otherwise Import/Export will likely still succeed but will automatically choose a different format.

This writes a whitespace separated plain text file that is readable by may other programs than Mathematica. If your dataset is so large that import/export takes too long, let me know, as there are better formats for that situation.

Answer 4

It is very easy. Save your file in .csv format e.g. Export["file.csv", data];, where data is like, data={{1.,1.},{2.,2.},{..}}. Then,during later use, initialize a list like, c={{0.,0.}} and then c = Import["file.csv"].