9
$\begingroup$

I have a notebook ("notebook 1") which in the beginning sets some parameters, does something depending on their value and saves the result to a file, say:

p=5;
result=p^2;
result>>"result";

I want to run this for many combinations of parameter values, so I'd like to automate the process. The notebook is large and I don't want to pack it into a single function of the parameters, as that would reduce readability.

So I want to run it from another notebook ("notebook 2"), effectively as if it were a function. I also want each run to be on a fresh kernel.

This is partly done by the following code

runWith[pval_]:=(
p=pval;
Protect[p];
NotebookEvaluate[notebookaddress];
Unprotect[p];
ClearAll[p])

where notebook 1 uses a parameter named p and this code runs it with value pval assigned to it. The protect is necessary because the notebook tries to set it to another value, as in the example above. However this keeps the kernel on successive runs.

Adding Quit[] to notebook 1 doesn't work, as it is run in the same kernel, so this will quit the kernel of notebook 2 as well.

A solution would be to let notebook 1 evaluate in another kernel, and then quit that kernel, both from notebook 2, but I can't find if this is possible.

$\endgroup$
  • 1
    $\begingroup$ one approach may be to run notebook1 from the command line via Run ( how this works is likely system dependent ). $\endgroup$ – george2079 Sep 29 '15 at 21:12
  • 1
    $\begingroup$ The problem you witness might be specific to the content of notebook 1. You should provide a minimal example for notebook 1, that shows this behavior. $\endgroup$ – Karsten 7. Sep 29 '15 at 21:19
  • $\begingroup$ You could adapt something from this: stackoverflow.com/a/7428340/879601 $\endgroup$ – Chris Degnen Oct 8 '15 at 19:50
  • $\begingroup$ What version of Mathematica are you using? I have a method that worksin 10.3 or later but not in 10.0 or earlier (didn't try 10.1 and 10.2). Do you insist on notebooks? Why not put everything into a package file (plain text .m file) and evaluate that with Get? It will save you a lot of trouble. $\endgroup$ – Szabolcs Dec 15 '16 at 12:31
  • $\begingroup$ @Szabolcs, one reason to work from notebooks directly vs packages is convenience. When you edit an API notebook daily, why go through the process of re-creating a pgk each time? - That's nothing but trouble. $\endgroup$ – alancalvitti May 10 '17 at 21:38
8
$\begingroup$

The best way is to use bash script!

First you need to save your script as .m file and "save as" option, wont work alone! You have to convert it to "code" format, by select all cmd+8 in OS X if I'm not wrong or select all and then fromat->style->code.

Second, we tell mathematica to run its kernel through a command. First we give the path to the Mathematica to run the kernel, which will run your script! Your code will look like below, the part which says result=p^2 sends the results value to the .m script which will be evaluated in the kernel.

SetDirectory["/Users/ray/Desktop"];
p = 4;
Run[Evaluate[
  "/Applications/Mathematica.app/Contents/MacOS/MathKernel -run \
results=\"" , p^2, "\"", "-script test.m > itest.txt"]]

your results will be saved in itest.txt you can easily open that in mathematica and process the results. You need complete path to .m files by the way, pay attention to setdirectory!

$\endgroup$
  • $\begingroup$ Thanks. I was hoping for something which works in mathematica itself, and which allows me to keep 'notebook 1' as it is, so i can run it manually as well. Do you think this isn't possible? $\endgroup$ – Jansen Oct 1 '15 at 18:54
  • $\begingroup$ This whole code you just write it in mathematica. You can run it anyhow you want it doesn't matter. why did you think it's not running inside mathematica? $\endgroup$ – Raymond Ghaffarian Shirazi Oct 3 '15 at 16:01
  • $\begingroup$ Sorry, I guess I didn't read carefully. I don't understand what this does though. Up until -run, I guess this starts up a kernel, a different one from the one the notebook this code is in is running? test.m is what i save my 'notebook 2' as? Then you send p^2 to this script somehow, but I want p itself as an input to the script (overriding any value it may give itself, like my code does above). And i'd like it to save and load the results with put and get instead of just to a text file. $\endgroup$ – Jansen Oct 7 '15 at 20:52
  • $\begingroup$ set directory should be the address to the folder where your file is, yes test.m is your notebook file saves as .m pay attention how to do it as explained above, then you can just put p instead of p^2, when you run the script the value of results is set to value of you can modify what is up there as '\ output=\"" , p' then you have a variable output set to the value of p in your script. you can open the text file with a simple command, maybe even have the results right there, though I haven't done it before. $\endgroup$ – Raymond Ghaffarian Shirazi Oct 7 '15 at 21:19
  • $\begingroup$ Ok, it's not quite what I was looking for, but thanks for the help. $\endgroup$ – Jansen Oct 8 '15 at 19:14
4
$\begingroup$

The best solution I have been able to come up with so far has the following properties:

  • Nothing needs to be modified in the 'worker' notebook (the one that's run many times), so it can be run manually as well without any changes.
  • The 'distributing' notebook, the one telling the 'worker' to do the runs, can override value assignments in the worker notebook.
  • Each run of the worker starts with a clean slate, previous runs can't interfere with the current one.
  • Any printing in the worker notebook is suppressed, so the distributing notebook is not flooded with prints.
  • Error messages still show though, and so do any Monitor commands.
  • Saving is done inside the worker notebook, with Put, making it easy to access.

The solution is as follows,

runWith[pValue_Integer] := (
Begin["WorkerContext`"];
p = pValue;
Protect[p];
timeTaken = 
Block[{System`Print = dontPrint, 
  System`StylePrint = StyleDontPrint}, 
 Quiet[NotebookEvaluate[NotebookDirectory[] <> "worker.nb"],Set::wrsym]
    //Timing // First];
Print["p = "<>ToString[p]<>" completed in " <> ToString[timeTaken] <> " s."];
Unprotect[p];
Unprotect["WorkerContext`*"];
ClearAll["WorkerContext`*"];
Remove["WorkerContext`*"];
End[];);

So starting with a clean slate is done by working in a new context, and clearing that context after each run. Overriding value assignments is done just by protecting them (I suppress the warning message this generates with Quiet), printing is suppressed by just redefining Print in a Block.

The 'worker' notebook would be as in my question (except that it would save to "results"<>ToString[p]).

One functionality which this does not have is to be easily parallelizable. If I run ParallelTable[runWith[p],{p,1,10}] it complains with FrontEndObject::notavail: A front end is not available; certain operations require a front end.

If anyone has a solution which has all of these properties and is also able to run in ParallelTable, and is not significantly more complicated, I will accept their answer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.