Is it possible to clone the state of a kernel into a second kernel for long evaluations?

Question

I often have a long calculation I want to execute, while still being able to run short evaluations to experiment. I've set up a second kernel for long evaluations and currently copy all the previous commands over, execute them, and then start the long evaluation. Obviously this is not ideal.

Is there a way to copy over all the function and variable definitions over and just have an exact replica of the current kernel?

Answer 1

Here's a quick and dirty way to save kernel state. First try to get the packages your have built into the system. This is obviously an incomplete list and doesn't include new paclets or paclets you download, etc, but it's a start:

Quit

Quiet[Get /@ Contexts[]];

$contexts =
  Flatten[
   {
      #,
      # <> "Private`",
      # <> "PackagePrivate`",
      # <> "PackageScope`"
      } & /@ Contexts[]
   ];

$contextStateFile = 
  FileNameJoin@{$TemporaryDirectory, "std_contexts.mx"};
Export[
  $contextStateFile,
  $contexts
  ];

Then use DumpSave to dump new contexts:

$statefile =
  FileNameJoin@{
    $TemporaryDirectory,
    "kernel_state.mx"
    };
statedump[] :=
  With[{
    conts =
     DeleteCases[Contexts[],
      Alternatives @@
       DeleteCases[
        Replace[$contexts, {
          Except[_List] :>
           ($contexts = 
             Get@FileNameJoin@{$TemporaryDirectory, "std_contexts.mx"})
          }],
        $Context]
      ]
    },
   DumpSave[
    $statefile,
    conts
    ]
   ];

Then I'll load my main package:

In[9]:= << BTools`

In[10]:= statedump[] // AbsoluteTiming

Out[10]= {0.112649, {"BTools`", "BTools`Private`Package`", "Global`", 
  "GridDump`", "Image`InteractiveImageDump`", 
  "NotebookTools`ControlsDump`", "System`Convert`DumpDump`", 
  "System`Convert`PackageDump`"}}

You'll notice it's pretty fast. But that's because not a lot got dumped:

In[11]:= $statefile // FileByteCount

Out[11]= 368906

If you've made more definitions it'll be slower.

Then I can quit and try reloading:

Quit

In[1]:= Get@FileNameJoin@{
   $TemporaryDirectory,
   "kernel_state.mx"
   }

In[2]:= $contexts // Length

Out[2]= 4936

And also the old contexts were loaded:

In[3]:= DeleteCases[Contexts[],
 Alternatives @@
  DeleteCases[
   Replace[$contexts, {
     Except[_List] :>
      ($contexts = 
        Get@FileNameJoin@{$TemporaryDirectory, "std_contexts.mx"})
     }],
   $Context]
 ]

Out[3]= {"BTools`", "BTools`Private`Package`", \
"Explore`ExploreDump`", "Global`", "GridDump`", \
"Image`InteractiveImageDump`", "NotebookTools`ControlsDump`", \
"PredictionStartupDump`", "System`Convert`DumpDump`", \
"System`Convert`PackageDump`"}

This won't properly cache all state but it's a start. An alternative is to define set of contexts to store, too.

Answer 2

If you have some function or other object that takes a long time to get ready such as:

f[x_] = x;

You can

Export["file.m", Definition[f]]
Exit

And then start new kernel with

Import["file.m"]
f[2]

2