6
$\begingroup$

Bug introduced in 8.0 or earlier and fixed in 9.0.0

I'm having trouble with parallel evaluation of compiled functions. Here is a simple example illustrating the problem:

testwm = Compile[ {{x, _Real}, {n, _Integer}},
    Module[ {sum, inc}, sum = 1.0; inc = 1.0; 
    Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum], 
   CompilationTarget -> "WVM"];

Map[(testwm[1.5, 2000000] + $KernelID) &, 
  Range[2 $ProcessorCount]] // AbsoluteTiming
(*
==> {1.4290817, {4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 
  4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169}}
*)
ParallelMap[(testwm[1.5, 2000000] + $KernelID) &, 
  Range[2 $ProcessorCount]] // AbsoluteTiming
(*
==> {1.4210813, {10.4817, 10.4817, 9.48169, 9.48169, 8.48169, 
  8.48169, 7.48169, 7.48169, 6.48169, 6.48169, 5.48169, 5.48169}}
*)
ParallelEvaluate[testwm = Compile[ {{x, _Real}, {n, _Integer}},
        Module[ {sum, inc}, sum = 1.0; inc = 1.0; 
      Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum], 
     CompilationTarget -> "WVM"];];
ParallelMap[(testwm[1.5, 2000000] + $KernelID) &, 
  Range[2 $ProcessorCount], DistributedContexts -> None] // AbsoluteTiming
(*
==> {0.2760158, {10.4817, 10.4817, 9.48169, 9.48169, 8.48169, 
  8.48169, 7.48169, 7.48169, 6.48169, 6.48169, 5.48169, 5.48169}}
*)
ParallelEvaluate[ClearAll[testwm]];

From this example it's clear that DistributeDefinitons is not working properly, since ParallelMap offered no speed-up. It does not help even if I add an explicit call to DistributeDefinitons. If I compile the function on each kernel, then I get the expected speed increase.

Is this (another) bug or am I doing something wrong?

I'm using mma 8.0.1 on win7 64 bit.

PS: for non compiled functions everything works as expected:

testnc[x_, n_] := 
  Module[ {sum, inc}, sum = 1.0; inc = 1.0; 
   Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum];
Map[(testnc[1.5, 10000] + $KernelID) &, 
  Range[2 $ProcessorCount]] // AbsoluteTiming
(*
==> {0.5660324, {4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 
  4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169}}
*)
ParallelMap[(testnc[1.5, 10000] + $KernelID) &, 
  Range[2 $ProcessorCount]] // AbsoluteTiming
(*
==> {0.1210069, {10.4817, 10.4817, 9.48169, 9.48169, 8.48169, 
  8.48169, 7.48169, 7.48169, 6.48169, 6.48169, 5.48169, 5.48169}}
*)
Improve this question
$\endgroup$
6
  • $\begingroup$ I can't reproduce your timing difference. The very first time you run ParallelMap, there is a time lag to launch the kernels. After that, I get the speed-up from ParallelMap of the kind that I would expect (roughly 5-fold on 6 cores). I am on Win 7 64 bit M8.0. $\endgroup$
    – Leonid Shifrin
    Jun 1, 2012 at 16:56
  • $\begingroup$ if I try DistributeDefinitions[testwm];ParallelTable[{$KernelID, OwnValues[testwm]}, {i, 1, 4}], it appears that the definitions do get distributed $\endgroup$
    – acl
    Jun 1, 2012 at 17:03
  • $\begingroup$ @acl Not on my machine. I ran this on a fresh kernel. Do you have 8.0.1? $\endgroup$
    – Ajasja
    Jun 1, 2012 at 17:11
  • $\begingroup$ 8.0.1, yes. I have the same timing as you do, I was simply saying that if you do what I say, it does appear that the slave kernels know about the functions. Also t = Compile[{}, $KernelID]; ParallelTable[t[], {i, 1, 4}] $\endgroup$
    – acl
    Jun 1, 2012 at 17:20
  • 1
    $\begingroup$ I can confirm your findings on 8.0.4. DistributeDefinitions doesn't seem to like CompiledFunctions: after compiling, try DistributeDefinitions[testwm]; ParallelEvaluate@HoldForm[testwm // Evaluate] and you'll see the slave kernels still don't know about testwm. In contrast, With[{testwmcopy = testwm}, ParallelEvaluate[testwm = testwmcopy]]; works. Don't have time to look into this now but I suspect a bug in DistributeDefinitions (it wouldn't be the first...). FWIW behaviour in version 7 is as it should be. $\endgroup$
    – Oleksandr R.
    Jun 1, 2012 at 17:31

1 Answer 1

Reset to default
2
$\begingroup$

Here's a quick workaround, so that one does not have to manually track which CompiledFunctions to distribute.

ClearAll[CompiledFunctionNames];
CompiledFunctionNames[pattern_] := 
  Select[Names["Global`*"], 
   MatchQ[Evaluate[Symbol@#], _CompiledFunction] &];

ClearAll[DistributeCompiledFunction]
DistributeCompiledFunction[name_] := 
  With[{copy = Symbol[name]}, 
   ParallelEvaluate[
    If[ MatchQ[Evaluate[Symbol@name], _Symbol],(*Print["Setting ",
     name];*)Evaluate[Symbol[name]] = copy]]];

ClearAll[DistributeCompiledFunctions];
DistributeCompiledFunctions[pattern_] := 
  DistributeCompiledFunction /@ CompiledFunctionNames[pattern];

and some rudimentary tests:

testwm = Compile[{{x, _Real}, {n, _Integer}}, 
   Module[{sum, inc}, sum = 1.0; inc = 1.0;
    Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum], 
   CompilationTarget -> "WVM"];
testwm1 = 
  Compile[{{x, _Real}, {n, _Integer}}, 
   Module[{sum, inc}, sum = 1.0; inc = 1.0;
    Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum], 
   CompilationTarget -> "WVM"];
a = 5; (*a non compiled symbol -- will not get destributed*)
(*see which compiled functions are in the current context*)
CompiledFunctionNames["Global`*"] 
DistributeCompiledFunctions["Global`*"];
(*distributing twice does not override previous definition*)
DistributeCompiledFunctions["Global`*"]; 
(*check if it worked*)
DeleteDuplicates@ParallelEvaluate@HoldForm[testwm // Evaluate]
DeleteDuplicates@ParallelEvaluate@HoldForm[testwm1 // Evaluate]
DeleteDuplicates@ParallelEvaluate@HoldForm[a // Evaluate]
ParallelMap[(testwm[1.5, 2000000] + $KernelID) &, 
      Range[2 $ProcessorCount], 
  DistributedContexts -> None] // AbsoluteTiming
ParallelEvaluate[ClearAll[testwm, testwm1]];

Disclaimer: I have not yet tested this extensively.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.