22
$\begingroup$

I run a large computation in a loop. At each cycle of the loop I produce large arrays that I save on a disk. At each cycle I would like to Quit Mathematica and then restart automatically, loading necessary files from the disk. I need this to manage memory usage. I tried various tricks but I do not think anything short of Quit will work in my case. Please let me know if there is a way to do it. I do not want to use one of the kernels to control the Quit process because I want to use all kernels for parallel computation and clear up memory everywhere at each cycle. Thank you.

$\endgroup$
  • 1
    $\begingroup$ If you want to restart the Kernel you can use this trick: 82803. For restarting Mathematica, there is a shortcut introduced in Shortcuts` Ctrl+R based on NetLink (probably). $\endgroup$ – Kuba Nov 19 '16 at 18:49
20
$\begingroup$

You could launch a different kernel and use that to run the computation. You will be controlling this "slave kernel" from another Mathematica session. This will allow you to script even quitting and restarting the slave kernel.

Using parallel tools

This is simpler and I recommend trying this approach first.

Launch a single kernel:

kernel = LaunchKernels[1]

Evaluate some code on that kernel:

ParallelEvaluate[
  mycode, kernel, 
  DistributedContexts -> None (* for M10.4 and later, prevents copying symbols from main kernel to subkernel *)
]

Quit it:

CloseKernels[kernel] (* close the kernel, like "Quit" *)

Also see ParallelSubmit, which will make it easier to launch several different computations which may run in parallel.

Using raw MathLink

If for some reason you don't want to use the parallel tools, you can use MathLink functions directly. This is more difficult as more things can go wrong. You will need to practically re-implement some things that the parallel tools already have. Be prepared to spend some time if you go this route ... but I am sure that it is completely doable (I am working on something like this at the moment to be able to control one version of Mathematica from another).

Launch a new kernel:

link = LinkLaunch[
  "\"" <> First[$CommandLine] <> "\"" <> " -mathlink"
]

I had to add the quotation marks because on my system there is a space in Mathematica's path. I am on OS X. I am not sure what is the proper way to handle this on Windows.

Verify it:

LinkReadyQ[link]
(* True *)

This returns True if there is something on the link that can be read. When a new kernel is started up, it will always send an In prompt, so we expect True after a successful startup.

If you package these steps into a function, it is important to know that LinkReadyQ will not return True untl there is something written on the other side of the link. This may take some time, afer all the other kernel needs time to initialize. It may be better to go straight to LinkRead, which will wait until input arrives.

Read the input prompt from the link:

LinkRead[link]
(* InputNamePacket["In[1]:= "] *)

Send an evaluation (warning: EvaluationPacket is not HoldAll!):

LinkWrite[
  link,
  Unevaluated@EvaluationPacket[1+1]
]

Read all response packets and print them:

While[True,
  result = LinkRead[link, HoldComplete];
  Print[result];
  If[MatchQ[result, HoldComplete[_ReturnPacket], Break[]]
]

(* HoldComplete[ReturnPacket[2]] *)

In practice you would do something else with the responses than print them. Note that the response may be something else than a ReturnPacket, such as MessagePacket, TextPacket, etc. There may be more than one response to a single evaluation (e.g. several messages, Printed things, etc.) The last response, however, should be a ReturnPacket.

The following will cause the kernel on the other end of the link to quit:

LinkClose[link]

References:

$\endgroup$
  • $\begingroup$ Can I use parallel approach to restart the kernel which was automatically launched by FE? $\endgroup$ – Kuba Apr 19 '17 at 13:49
9
$\begingroup$

On Windows you can run your cycles in a batch file like this:

@echo off
setlocal
PATH = C:\Program Files\Wolfram Research\Mathematica\7.0\;%PATH%

echo Launching MathKernel %TIME%
break > "C:\Temp\Incomplete"
start MathKernel -noprompt -initfile "C:\Temp\Cycle1.m"
:Label1
ping localhost -n 5 > nul
if exist "C:\Temp\Incomplete" goto Label1
ping localhost -n 1 > nul
echo Terminating MathKernel %TIME%
tskill MathKernel > nul
ping localhost -n 2 > nul

echo Launching MathKernel %TIME%
break > "C:\Temp\Incomplete"
start MathKernel -noprompt -initfile "C:\Temp\Cycle2.m"
:Label2
ping localhost -n 5 > nul
if exist "C:\Temp\Incomplete" goto Label2
ping localhost -n 1 > nul
echo Terminating MathKernel %TIME%
tskill MathKernel > nul
ping localhost -n 2 > nul

endlocal

The batch file will run Cycle1.m, Cycle2.m etc. This is what Cycle1.m should contain in order to run Notebook1.nb entirely:

NotebookPauseForEvaluation[nb_] := Module[{},
While[NotebookEvaluatingQ[nb],Pause[.25]]];

NotebookEvaluatingQ[nb_]:=Module[{},
SelectionMove[nb,All,Notebook];
Or@@Map["Evaluating"/.#&,Developer`CellInformation[nb]]
];

UsingFrontEnd[
nb = NotebookOpen["C:\\Temp\\Notebook1.nb"];
SetOptions[nb, Evaluator -> "Kernel2"];
SelectionMove[nb, All, Notebook];
SelectionEvaluate[nb]
NotebookPauseForEvaluation[nb];
NotebookSave[nb];
DeleteFile["C:\\Temp\\Incomplete"];
];

Quit[];

Similarly Cycle2.m should be set to evaluate Notebook2.nb.

Note. To avoid intermittent clashes a second kernel, e.g. Kernel2, is required. It can be set up via the Evaluation menu, Kernel Configuration Options, Add, Kernel Name: Kernel2, OK.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.