Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I have a couple of big lists (each of them contains elements that are themselves lists of two elements, i.e. an element of each list that I have is of the form {x, y}) that I want to save/export. Their generation namely takes hours, and I don't want to do this every single day.

I looked at the Mathematica help section on this (http://reference.wolfram.com/mathematica/tutorial/ImportingAndExportingData.html), but found that if I follow the example, I can't just import the data back into a list as it was before. It just ends up being something really messy.

So given such a list, say, list = {{1,2}, {1,3}, ... , {500, 500}}, what do I do, so that the next day I can just write list = Import[...]?

share|improve this question
1  
I'd use .mx files (Export / Import in "MX" format). This is fast, and does not really involve serialization / parsing in the usual sense (via strings). In other words, mx files bypass the high-level parsing, populating internal structures at lower level. In addition, mx files preserve packed arrays. –  Leonid Shifrin Jun 24 at 21:20
    
@LeonidShifrin, yes! This works perfectly! If you make it an answer, I'd be glad to accept it. How did you figure this out, by the way? –  Ryker Jun 24 at 21:55
    
Actually, I recall now that I first learned about the fact that Export / Import working on .mx files from @Szabolcs. –  Leonid Shifrin Jun 24 at 22:06
1  
I find this Q&A relevant and handy. –  Johu Jun 25 at 0:15
    
@Johu, thanks, I'll take a closer look it when I have the time. –  Ryker Jun 25 at 2:53

3 Answers 3

up vote 4 down vote accepted

I'd use .mx files (Export / Import in "MX" format):

Export["myFile.mx",list]

and

Import["myFile.mx"]

This is fast, and does not really involve serialization / parsing in the usual sense (via strings). In other words, mx files bypass the high-level parsing, populating internal structures at lower level. In addition, mx files preserve packed arrays.

share|improve this answer
2  
I didn't mention MX on purpose because I thought it was overkill for relatively small strictly tabular data, and it does have some dangers that are important to be mentioned: it's not cross platform (before v10) and it's not compatible between different versions of Mathematica. I'm commenting just to warn the OP and make sure he doesn't use it for archiving or doesn't try to move files between say, Windows and Linux. –  Szabolcs Jun 24 at 22:06
    
@Szabolcs Yep, I do realize. Just when you posted this one, I added a comment under the main question, acknowledging that I learned about Export / Import working on .mx files from you. In this particular case, however, the OP's goal seems to be saving data for later use on the same machine, thus this suggestion. –  Leonid Shifrin Jun 24 at 22:08
    
Might comment was really aimed at the OP, not you. Yes, I agree than in general for saving important session data MX is best. Some systems like R even have a feature to restore the complete workspace, this is similar to that. –  Szabolcs Jun 24 at 22:11
    
@Szabolcs I disagree however that this is an overkill w.r.t. exporting as Table. The latter involves high-level serialization / parsing, and that always increases chances to not get the same thing back. Serializing to a binary format like .mx is different. –  Leonid Shifrin Jun 24 at 22:12
    
@Szabolcs Ok, apparently we agree on all points. –  Leonid Shifrin Jun 24 at 22:13

I provide two ways:

1) Human readable

data1 = RandomInteger[100, {25, 25}];
data2 = RandomReal[100, {25, 25}];
Save["humanReadable.m", {data1, data2}];

Unset[{data1, data2}]
Get["humanReadable.m"];
Dimensions@{data1, data2}

{2, 25, 25}

Note, that you can dump many different variables with ease, and the file is in easy to read Mathematica syntax allowing all kinds of symbolic and numeric data without any manual serialisation. By default Save appends which can be convenient, but ofc. must be kept in mind.

Saving the data from other sources in such format might be very handy, as you can use all of the Mathematica syntax including comments. For example I use it for measurement control software data dump.

2) Platform independent binary

Obviously the upside of binary format is smaller file size and loading time in case of big data.

In addition to mx data format already discussed, there is version and platform independend format wdx. Again there is a way to export and import it with symbol names attached.

DumpSave["platformIndependendBinary.wdx", {data1, data2}];
Unset[{data1, data2}]
Get["platformIndependendBinary.wdx"]

And if you don't want to fix / remember the variable names you can

DeleteFile["platformIndependendBinary.wdx"]
Export["platformIndependendBinary.wdx", {data1, data2}]
Unset[{data1, data2}]
{data1, data2} = Import["platformIndependendBinary.wdx"];

The same method works for .m files for human readable text format.

The only downside of wdx compared to mx is speed.

share|improve this answer
4  
WDX is awfully slow though. I use Export[..., Compress[expr], "String"], which is also version/platform independent, and faster than WDX. –  Szabolcs Jun 25 at 0:20
    
+1 for nice tip. The discussion about the speed I already stumbled upon and linked. –  Johu Jun 25 at 0:21

Probably the best way is to do

Export["mydata.txt", list, "Table"]

then later

Import["mydata.txt", "Table"]

Be sure to explicitly specify the data format: "Table". Otherwise Import/Export will likely still succeed but will automatically choose a different format.

This writes a whitespace separated plain text file that is readable by may other programs than Mathematica. If your dataset is so large that import/export takes too long, let me know, as there are better formats for that situation.

share|improve this answer
    
I just tried it, but it doesn't work as I want it to. I namely don't get the original list if I do Import["mydata.txt"]. –  Ryker Jun 24 at 21:53
    
@Ryker For a list of the form you mentioned, it always works for me. Please post a short example list you have, then we can figure out why we see different behaviour. –  Szabolcs Jun 24 at 22:03
    
Well, for example, list = {{1,2}, {1,3}}. –  Ryker Jun 24 at 22:28
1  
@Ryker dropbox.com/s/ichx99uq4sdlujv/… –  Szabolcs Jun 24 at 22:29
    
Damn, you're right, I overlooked the "Table" part in Import... –  Ryker Jun 24 at 22:34

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.