Friendly way to make previous large calculation results reusable in a new session

Very often, I need to do same calculations under different parameters for comparing the results. Every calculation may generate a large data set; for example, the calculation result of Eigensystem is often a large list.

If we want to make a careful systematic investigation of the whole parameter space, the best way is to export calculation result into external file.

But before we make a careful systematic investigation of the whole parameter space, we usually need to look into several typical parameter cases to see whether the result is consistent with my original conjecture or not. Under these circumstances, my personal experience is that exporting and importing is not as flexible as directly leaving results in the notebook.

Then here is the issue of how to store results from large calculations compactly in the notebook and how to obtain the results without a re-calculation when I reopen the notebook to continue my previous analysis.

For example:

The following assigns an eigensystem to variable es If I end my Mathematica session and then reopen the same notebook in a new season, but don't want to re-calculate, then I have to do an assignment like below But in this way, the input cell is way too large! I don't know how to hide it. And this way is just too awkward!

One possible way is to seek method to pack the large output of calculation into something that Dispatch would generate like the following. I found the dispatch object is reusable when reopen mma.

I also wish to get any advice that might be forthcoming about properly organizing calculations which give results intended for use in futher analysis?

• "the input cell is way too large!!" - consider dividing your notebook into sections, so that any section not being actively looked at can be folded into one cell. – J. M. will be back soon Oct 21 '15 at 6:02
• As of V10.2, you can use local objects to deal with such problems. Read about how local objects can be used to maintain data that has to persist over sessions by following this link – m_goldberg Oct 21 '15 at 6:28
• I agree with "I like the flexibility in notebook". I often use J.M.'s method, where I have data in a cell which is under a heading entitled Run this first, and you can collapse all the cells together. m_goldberg's solution sounds interesting, too, but since I don't have 10.2, I can't comment. – march Oct 21 '15 at 6:34
• You can use TaggingRules, just put CurrentValue[EvaluationNotebook[], {TaggingRules, "varname"}] = es. End you can always get it, it will be stored with the notebook. Take a look at related: 88343 – Kuba Oct 21 '15 at 6:50
• Sometimes the best approach is to export these results into a file, then re-import them later. This is also related: mathematica.stackexchange.com/q/11891/12 – Szabolcs Oct 21 '15 at 7:57

LocalSymbol lets you persistently save objects to your filesystem.

LocalSymbol["x"] = (Pause; 6);
Quit[]

then restart the kernel and run

LocalSymbol["x"]

The output is an immediate 6.

Disclaimer: If used with large expressions, wrapping it in the described way introduces some kind of overhead on saving the notebook. Also, I haven't tested this extensively, there might be other problems!

To create a "folding" functionality akin to what you get from the Dispatch box for example, we can take a look at this answer. Using the technique described there we can create a wrapper to hide the contents of any expression:

label[lbl_, v_] := Row[{Style[lbl <> ": ", "SummaryItemAnnotation"], v}];

iconize /: MakeBoxes[ifun : iconize[s1_], fmt_] :=
BoxFormArrangeSummaryBox[Head@s1, ifun,
"", {label["LeafCount", LeafCount@s1]},
{label["Content", OutputSizeLimitShort@s1]}, fmt];

Using this we can then do something like

ls = iconize@Range
ListPlot@ls[]

where in the second line the data has to be unwrapped from the iconize.

Note that automatic unwrapping can be achieved by the additional definition

iconize /: (h : Except[Set])[t1___, iconize[s1_], t2___] := h[t1, s1, t2]

However, this introduces a strange behaviour when copy/paste-ing the output.

• Would you mind to post an image of the result? Because I can't get wrapped result in v10.2 – matheorem Oct 21 '15 at 13:52
• Thank you very much. I tried it, but mma got frozen when I tried to save the notebook, have no idea why. What is more, it takes significantly more time to generate the object. – matheorem Oct 21 '15 at 15:38
• @matheorem I can't reproduce either of these problems. What did you use as input data? – sebhofer Oct 21 '15 at 16:23
• Ups, I forgot to give an example. On my computer, iconize[Eigensystem@RandomReal[{0,1},{500,500}]] will freeze mma when saving. If it not in your computer, maybe try {1000,1000}, it is just an eigensystem of 1000x1000 matrix – matheorem Oct 22 '15 at 0:43
• @matheorem I can reproduce the crash on saving the notebook. This can be improved by shortening long output in the object (see updated code). There is still an overhead on saving large data. I really don't know what is going on there though. – sebhofer Oct 22 '15 at 11:57

Use of tagging rules as mentioned by Kuba.

es = Eigensystem[RandomReal[{0, 1}, {100, 100}]];

SetOptions[EvaluationNotebook[], TaggingRules -> {"varname" -> es}]

NotebookSave[]

Quit[]

In a new session the data can be recovered like so.

es = CurrentValue[{TaggingRules, "varname"}]
• Have you tried Eigensystem[RandomReal[{0, 1}, {1000, 1000}]], in my computer this froze mma and there is no sign of recovering, so I have to shut it down abnormally. – matheorem Oct 23 '15 at 12:56
• It works fine on Mathematica 7.0.1.0. It takes about 10 seconds to save the notebook and the notebook ends up nearly 60 MB large. It hangs Mathematica 10.1.0.0 on saving. Mathematica 10.1.0.0 also hangs if it tries to do anything with the saved version 7 notebook. – Chris Degnen Oct 23 '15 at 13:33
• A bug in Mathematica 10? – matheorem Oct 23 '15 at 14:07
• And the size is another problem, LocalSymbol for eigensystem of 1000x1000 will have 49MB. If zip it, it reduced to 17MB. I feel that mathematica takes too much space. – matheorem Oct 23 '15 at 14:13
• You could store Compress[es] – Chris Degnen Oct 23 '15 at 17:21