6
$\begingroup$

Bug introduced in 8.0 or earlier and fixed in 9.0.0


I'm having trouble with parallel evaluation of compiled functions. Here is a simple example illustrating the problem:

testwm = Compile[ {{x, _Real}, {n, _Integer}},
    Module[ {sum, inc}, sum = 1.0; inc = 1.0; 
    Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum], 
   CompilationTarget -> "WVM"];

Map[(testwm[1.5, 2000000] + $KernelID) &, 
  Range[2 $ProcessorCount]] // AbsoluteTiming
(*
==> {1.4290817, {4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 
  4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169}}
*)
ParallelMap[(testwm[1.5, 2000000] + $KernelID) &, 
  Range[2 $ProcessorCount]] // AbsoluteTiming
(*
==> {1.4210813, {10.4817, 10.4817, 9.48169, 9.48169, 8.48169, 
  8.48169, 7.48169, 7.48169, 6.48169, 6.48169, 5.48169, 5.48169}}
*)
ParallelEvaluate[testwm = Compile[ {{x, _Real}, {n, _Integer}},
        Module[ {sum, inc}, sum = 1.0; inc = 1.0; 
      Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum], 
     CompilationTarget -> "WVM"];];
ParallelMap[(testwm[1.5, 2000000] + $KernelID) &, 
  Range[2 $ProcessorCount], DistributedContexts -> None] // AbsoluteTiming
(*
==> {0.2760158, {10.4817, 10.4817, 9.48169, 9.48169, 8.48169, 
  8.48169, 7.48169, 7.48169, 6.48169, 6.48169, 5.48169, 5.48169}}
*)
ParallelEvaluate[ClearAll[testwm]];

From this example it's clear that DistributeDefinitons is not working properly, since ParallelMap offered no speed-up. It does not help even if I add an explicit call to DistributeDefinitons. If I compile the function on each kernel, then I get the expected speed increase.

Is this (another) bug or am I doing something wrong?

I'm using mma 8.0.1 on win7 64 bit.

PS: for non compiled functions everything works as expected:

testnc[x_, n_] := 
  Module[ {sum, inc}, sum = 1.0; inc = 1.0; 
   Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum];
Map[(testnc[1.5, 10000] + $KernelID) &, 
  Range[2 $ProcessorCount]] // AbsoluteTiming
(*
==> {0.5660324, {4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 
  4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169}}
*)
ParallelMap[(testnc[1.5, 10000] + $KernelID) &, 
  Range[2 $ProcessorCount]] // AbsoluteTiming
(*
==> {0.1210069, {10.4817, 10.4817, 9.48169, 9.48169, 8.48169, 
  8.48169, 7.48169, 7.48169, 6.48169, 6.48169, 5.48169, 5.48169}}
*)
$\endgroup$
  • $\begingroup$ I can't reproduce your timing difference. The very first time you run ParallelMap, there is a time lag to launch the kernels. After that, I get the speed-up from ParallelMap of the kind that I would expect (roughly 5-fold on 6 cores). I am on Win 7 64 bit M8.0. $\endgroup$ – Leonid Shifrin Jun 1 '12 at 16:56
  • $\begingroup$ if I try DistributeDefinitions[testwm];ParallelTable[{$KernelID, OwnValues[testwm]}, {i, 1, 4}], it appears that the definitions do get distributed $\endgroup$ – acl Jun 1 '12 at 17:03
  • $\begingroup$ @acl Not on my machine. I ran this on a fresh kernel. Do you have 8.0.1? $\endgroup$ – Ajasja Jun 1 '12 at 17:11
  • $\begingroup$ 8.0.1, yes. I have the same timing as you do, I was simply saying that if you do what I say, it does appear that the slave kernels know about the functions. Also t = Compile[{}, $KernelID]; ParallelTable[t[], {i, 1, 4}] $\endgroup$ – acl Jun 1 '12 at 17:20
  • 1
    $\begingroup$ I can confirm your findings on 8.0.4. DistributeDefinitions doesn't seem to like CompiledFunctions: after compiling, try DistributeDefinitions[testwm]; ParallelEvaluate@HoldForm[testwm // Evaluate] and you'll see the slave kernels still don't know about testwm. In contrast, With[{testwmcopy = testwm}, ParallelEvaluate[testwm = testwmcopy]]; works. Don't have time to look into this now but I suspect a bug in DistributeDefinitions (it wouldn't be the first...). FWIW behaviour in version 7 is as it should be. $\endgroup$ – Oleksandr R. Jun 1 '12 at 17:31
2
$\begingroup$

Here's a quick workaround, so that one does not have to manually track which CompiledFunctions to distribute.

ClearAll[CompiledFunctionNames];
CompiledFunctionNames[pattern_] := 
  Select[Names["Global`*"], 
   MatchQ[Evaluate[Symbol@#], _CompiledFunction] &];

ClearAll[DistributeCompiledFunction]
DistributeCompiledFunction[name_] := 
  With[{copy = Symbol[name]}, 
   ParallelEvaluate[
    If[ MatchQ[Evaluate[Symbol@name], _Symbol],(*Print["Setting ",
     name];*)Evaluate[Symbol[name]] = copy]]];

ClearAll[DistributeCompiledFunctions];
DistributeCompiledFunctions[pattern_] := 
  DistributeCompiledFunction /@ CompiledFunctionNames[pattern];

and some rudimentary tests:

testwm = Compile[{{x, _Real}, {n, _Integer}}, 
   Module[{sum, inc}, sum = 1.0; inc = 1.0;
    Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum], 
   CompilationTarget -> "WVM"];
testwm1 = 
  Compile[{{x, _Real}, {n, _Integer}}, 
   Module[{sum, inc}, sum = 1.0; inc = 1.0;
    Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum], 
   CompilationTarget -> "WVM"];
a = 5; (*a non compiled symbol -- will not get destributed*)
(*see which compiled functions are in the current context*)
CompiledFunctionNames["Global`*"] 
DistributeCompiledFunctions["Global`*"];
(*distributing twice does not override previous definition*)
DistributeCompiledFunctions["Global`*"]; 
(*check if it worked*)
DeleteDuplicates@ParallelEvaluate@HoldForm[testwm // Evaluate]
DeleteDuplicates@ParallelEvaluate@HoldForm[testwm1 // Evaluate]
DeleteDuplicates@ParallelEvaluate@HoldForm[a // Evaluate]
ParallelMap[(testwm[1.5, 2000000] + $KernelID) &, 
      Range[2 $ProcessorCount], 
  DistributedContexts -> None] // AbsoluteTiming
ParallelEvaluate[ClearAll[testwm, testwm1]];

Disclaimer: I have not yet tested this extensively.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.