3
$\begingroup$

Suppose I have a function f which takes a list and does something with it:

f[mylist_List]:=Block[{},
                tmp=Total[mylist];
                {Length[mylist],tmp}
                ];

(For illustration purposes the function above computes the sum of all elements, however this is just to keep the example simple and my actual application does something else.)

Assume that I have a large list ll and would like to apply the operation f to it. But while doing that, I would like to manually split ll into 8 parts and apply f to those parts in parallel on 8 different cores - i.e.:

largelist=Table[i,{i,1,80}];
ParallelExecute[f[ll[[1;;10]]],f[ll[[11;;20]]],f[ll[[21;;30]]],f[ll[[31;;40]]],f[ll[[41;;50]]],f[ll[[51;;60]]],f[ll[[61;;70]]],f[ll[[71;;80]]]]

While doing that I would have 8 Kernels running and the function ParallelExecute should perform the individual calculations on each of these Kernels separately, as if I had 8 different notebooks assigned to different Kernels open at the same time and did the calculations in them. Afterwards the routine should return the 8 separate results in a list to the original notebook from where ParallelExecute was called. (Preferably, Mathematica should not even be aware of the fact that a single routine is being parallelized, but think that 8 different processes are running on 8 different Kernels.)

I am aware of the existance of ParallelTable and similar routines. I am not happy with those solutions, since they are too much of a black box and from my experience take way too long to break the input into 8 pieces and distribute them between Kernels, that is why I would like to do this manually to speed things up. Also it would be nice to have a shared memory region in RAM where all Kernels have simultaneous access to.

Does ParallelExecute exist? Or maybe it can be implemented in a convenient fashion? Thanks for any suggestion!

EDIT:

Thanks to the response by bbgodfrey we can show that the manual way of parallelizing yields a huge speedup. Define the following site standard function to measure time averages

SetAttributes[timeAvg, HoldFirst]
timeAvg[func_] := Do[If[# > 0.3, Return[#/5^i]] & @@ Timing@Do[func, {5^i}], {i, 0, 15}]

Then we can put:

LaunchKernels[8];
ParallelExecute[myinput_List] := {ParallelEvaluate[f[myinput[[1]]],Kernels[][[1]]]
                                  ,ParallelEvaluate[f[myinput[[2]]],Kernels[][[2]]]
                                  ,ParallelEvaluate[f[myinput[[3]]],Kernels[][[3]]]
                                  ,ParallelEvaluate[f[myinput[[4]]],Kernels[][[4]]]
                                  ,ParallelEvaluate[f[myinput[[5]]],Kernels[][[5]]]
                                  ,ParallelEvaluate[f[myinput[[6]]],Kernels[][[6]]]
                                  ,ParallelEvaluate[f[myinput[[7]]],Kernels[][[7]]]
                                  ,ParallelEvaluate[f[myinput[[8]]],Kernels[][[8]]]}

And test

timeAvg[ParallelExecute[Table[largelist[[1 + 10 j ;; (j + 1) 10]], {j, 0, 7}]]]

0.00312500

versus

timeAvg[ParallelTable[ Total[largelist[[1 + 10 j ;; (j + 1) 10]]], {j, 0, 7}]]

0.00662500

$\endgroup$
  • $\begingroup$ You could call ParallelEvaluate[expr,kernel] eight times, one for each kernel. $\endgroup$ – bbgodfrey Apr 8 '16 at 18:27
  • $\begingroup$ Also, ParallelSubmit is possible. Although ParallelEvaluate seems simpler $\endgroup$ – Lukas Apr 8 '16 at 18:59
1
$\begingroup$

To make my comment more concrete (for a four-processor machine),

{ParallelEvaluate[f[largelist[[1 ;; 20]]], 1], 
 ParallelEvaluate[f[largelist[[21 ;; 40]]], 2], 
 ParallelEvaluate[f[largelist[[41 ;; 60]]], 3], 
 ParallelEvaluate[f[largelist[[61 ;; 80]]], 4]}
Total[%]

(* {{20, 210}, {20, 610}, {20, 1010}, {20, 1410}}
   {80, 3240} *)
$\endgroup$
  • $\begingroup$ This looks exactly as the function I am looking for. However, I still wonder how the memory is managed in this case? Do the different pieces get copied over to separate memory regions assigned to different Kernels, or do all Kernels have access to the single original to perform the computations? $\endgroup$ – Kagaratsch Apr 8 '16 at 18:46
  • 1
    $\begingroup$ @Kagaratsch Only specified functions or variables are shared. Look here for details. $\endgroup$ – bbgodfrey Apr 8 '16 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.