8
$\begingroup$

I have a notebook with several input cells that produce data which is then plotted into separate graphs.

Since I do not want to save the results, because it makes the file huge, I keep the unevaluated file and run 'evaluate notebook' to produce all the plots. The cells are not sharing anything and are independent, so they could theoretically be run on separate kernels. I can run them in separate notebooks on separate kernels of course, but how can I achieve it with one notebook?

I tried to find my answer, however most applications want to run a ParallelEvaluate. However, I would like to run each cell on a separate kernel, which means that the cells run in parallel, but each cell is only run by one single kernel (so one single cell is not multithreaded).

Metaphorically speaking, I do not want a parallel running for loop within one cell, but I want four different separate for loops running in separate cells and in parallel to each other. Tell me if I use any incorrect terms or if my explanation is lacking anything.

Improve this question
$\endgroup$
8
  • $\begingroup$ I think you may be able to schedule your various evaluations using ParallelSubmit and then start all the evaluations using WaitAll. See if you can adapt the examples in the ParallelSubmit docs. $\endgroup$
    – MarcoB
    Commented Apr 29, 2016 at 14:15
  • $\begingroup$ Does that mean I have to rewrite my code into one cell with a huge ParallelSubmit? Or can I somehow refer to the other cells and submit their content directly? Another idea I had was through the assignment of cells to a specific kernel, but what I would actually want to do is to ask for an idle kernel and then assign the cell to that one. However, I could not find a method that gives me the idleness of a kernel. $\endgroup$
    – Ben
    Commented Apr 29, 2016 at 14:21
  • $\begingroup$ You can try the Evaluator option of Cell which allows to attach individual kernels to individual cells in one notebook and run them in parallel. $\endgroup$
    – Alexey Popkov
    Commented Apr 29, 2016 at 23:07
  • $\begingroup$ Related: "Is there a method to send different Cell evaluation to different core?" $\endgroup$
    – Alexey Popkov
    Commented Apr 30, 2016 at 0:44
  • 1
    $\begingroup$ @Ben Please see this answer of mine for some examples of Palettes with such functionality. $\endgroup$
    – Alexey Popkov
    Commented May 7, 2016 at 9:37

1 Answer 1

Reset to default
11
$\begingroup$

You don't have to write all your code within ParallelSubmit. You can wrap your code in Module or Block and assign it to a variable using SetDelayed (:=) so it won't be evaluated immediately.

Here is a toy example:

eval1 := PrimeQ[528973465287364528736543]
eval2 := Module[{}, Pause[2]; 3]

evaluators = ParallelSubmit[{eval1, eval2}]

ready, not evaluated

You then run the submitted code using WaitAll. This returns a list containing the return values of each evaluator, in the order in which they were submitted, and also changes the state of the evaluator graphics representation to "finished"

WaitAll[evaluators]
(* Out: {False, 3} *)

done

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.