6
$\begingroup$

When I use ParallelSubmit, is it possible to select on which kernel a task can possibly be executed?

For example if have 3 parallel kernels, can I only use 2 of them so that one is remaining in case I need an emergency task to run on it? But my question is not just for 3 kernels.

This question is equivalent to "Is it possible to associate different kernels with different queues?".

$\endgroup$
  • $\begingroup$ Maybe worth looking at ParallelEvaluate? $\endgroup$ – Chen Stats Yu Aug 2 '15 at 22:19
  • 1
    $\begingroup$ ParallelEvaluate[expr,kernel] evaluates expr on the parallel kernel specified. $\endgroup$ – Chen Stats Yu Aug 2 '15 at 22:20
  • $\begingroup$ Yes but the main kernel waits for the result, thus the main kernel cannot be used at the same time, like when using ParallelSubmit. $\endgroup$ – faysou Aug 3 '15 at 7:41
4
$\begingroup$

First answer

It doesn't seem possible but ParallelSubmit has a Scheduling option which allows to set a priority to a task. The lower it is the earlier the task will be executed.

For example

LaunchKernels[1]
ParallelSubmit[2+2, Scheduling -> 5]

Maybe in case of needing to execute an emergency task on a parallel kernel, a solution would be to submit a task with a high priority (i.e. lower Scheduling than other tasks), launch a new kernel, do QueueRun so that the task gets executed, and then close the new kernel once the task is done, in order not to have a growing amount of kernels.


Second answer

A better possibility is to create a link without LaunchKernels and send to it expressions to evaluate. I've followed the idea of Szabolcs in this answer.

The only disadvantage I see, is that shared functions and variables won't probably be shared with the new kernel (maybe it's possible to get this done, I don't know yet).

kernel=LinkLaunch@"mathkernel -mathlink";
LinkRead[kernel]; (*Return the first input line*)

readFromLink[link_]:=
    If[LinkReadyQ@link,
        LinkRead[link]//First
        ,
        Missing["LinkNotReady"]
    ];

LinkWrite[kernel,Unevaluated[Pause@5; 2+2]];

For this example you can evaluate several times the expression below, or just wait a little bit. Note that the main kernel is free.

So you can evaluate to a specific kernel like with ParallelEvaluate, and the main kernel is free like with ParllelSubmit.

readFromLink@kernel

Then you can close the kernel.

LinkClose@kernel

A reference on these functions is here.


Third answer

Here's a third way that I ended up using. I looked at how the Parallel package is implemented and found a hack in order to remove some kernels from the list of kernels where queued evaluations can run. Thus I can still have a queue and kernels that are dedicated to some specific tasks and can be accessed immediately.

A bonus of this method is that reserved kernels can still be debugged in Wolfram workbench and have accesse to shared variables.

The ability to send an instruction to a specific kernel without blocking the main kernel also allows to build one's own queues.

LaunchKernels[3];
reservedKernel = First@Parallel`Protected`$sortedkernels;
Parallel`Protected`$sortedkernels=Rest@Parallel`Protected`$sortedkernels
Parallel`Developer`Send[reservedKernel,2+2]
Parallel`Developer`ReceiveIfReady[reservedKernel]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.