Button action preventing parallel execution?

Question

I couldn't find a better title, so I'll jump straight to the example!

There is this long list of slow jobs to be executed in a parallel way, at the press of a button:

DynamicModule[{dispatch, list, doStuff},
 list = Range[8];
 dispatch[] := ParallelMap[(Pause[0.5]; Print@{$KernelID, #}) &, list];
 Button["Go", dispatch[] , Method -> "Queued"]
 ]

This works as expected, with every kernel printing it's job and ID. As soon as I start modularizing code, things break:

DynamicModule[{dispatch, list, doStuff},
 list = Range[8];
 doStuff[a_] := (Pause[0.5]; Print@{$KernelID, a});
 dispatch[] := ParallelMap[doStuff, list];
 Button["Go", dispatch[] , Method -> "Queued"]
 ]

This executes sequentially using only the main kernel (ID 0).

I've tried distributing definitions, contexts and even sharing variables, but the closest I've gotten to parallelization was the following:

DynamicModule[{dispatch, list, doStuff, doSlowStuff},
 list = Range[8];
 doSlowStuff[] := Pause[0.5];
 doStuff[a_] := (doSlowStuff[]; Print@{$KernelID, a});
 dispatch[] := ParallelMap[doStuff, list];
 dispatch[];  (*This call is correctly executed in parallel*)
 Button["Go", dispatch[] , Method -> "Queued"]
 ]

The "inline" dispatch[] call is correctly executed (using all kernels) even with the added nested definition of doSlowStuff, but the button's dispatch[] still executes sequentially on kernel 0.

What am I missing?

Answer 1

It might be that this is due to dispatch, list and doStuff being completely owned by the frontend, since you wrapped them in a DynamicModule. Frontend variables cannot be shared between kernels. When you localize them in a Module, and therefore keep them as kernel variables, it works:

Module[{dispatch, list, doStuff}, list = Range[8];
 doStuff[a_] := (Pause[0.5]; Print@{$KernelID, a});
 dispatch[] := ParallelMap[doStuff, list];
 Button["Go", dispatch[], Method -> "Queued"]]

(* {4,1} {3,3} {2,5} {1,7} {4,2} {3,4} {2,6} {1,8} *)

It is possible to accomplish this task in a DynamcModule:

DynamicModule[{dispatch, doSlowStuff, list = Range[6]},
 doSlowStuff[x_] := (Pause[0.1]; x^2);
 dispatch[] := ParallelMap[Print[{$KernelID, doSlowStuff[#]}] &, list];
 Button["Go", SetSharedFunction[doSlowStuff, dispatch]; dispatch[]]
]

(* {4,1} {3,9} {2,25} {1,36} {4,4} {3,16} *)

The local frontend variables dispatch and doSlowStuff are linked to kernel variables in the context FE`. It seems that these variables are not automatically distributed over the sub kernels, so we have to use SetSharedFunctions. When this is done outside the button, the kernel variables and not the frontend variables are distributed. Since the kernel variables are removed when the evaluation of the DynamicModule is completed, that will not work. So SetSharedFunctions must be used inside the button.