DistributeDefinitions not working for compiled functions

This is fixed in Mathematica 9.0.0.

I'm having trouble with parallel evaluation of compiled functions. Here is a simple example illustrating the problem:

testwm = Compile[ {{x, _Real}, {n, _Integer}},
Module[ {sum, inc}, sum = 1.0; inc = 1.0;
Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum],
CompilationTarget -> "WVM"];

Map[(testwm[1.5, 2000000] + $KernelID) &, Range[2$ProcessorCount]] // AbsoluteTiming
(*
==> {1.4290817, {4.48169, 4.48169, 4.48169, 4.48169, 4.48169,
4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169}}
*)
ParallelMap[(testwm[1.5, 2000000] + $KernelID) &, Range[2$ProcessorCount]] // AbsoluteTiming
(*
==> {1.4210813, {10.4817, 10.4817, 9.48169, 9.48169, 8.48169,
8.48169, 7.48169, 7.48169, 6.48169, 6.48169, 5.48169, 5.48169}}
*)
ParallelEvaluate[testwm = Compile[ {{x, _Real}, {n, _Integer}},
Module[ {sum, inc}, sum = 1.0; inc = 1.0;
Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum],
CompilationTarget -> "WVM"];];
ParallelMap[(testwm[1.5, 2000000] + $KernelID) &, Range[2$ProcessorCount], DistributedContexts -> None] // AbsoluteTiming
(*
==> {0.2760158, {10.4817, 10.4817, 9.48169, 9.48169, 8.48169,
8.48169, 7.48169, 7.48169, 6.48169, 6.48169, 5.48169, 5.48169}}
*)
ParallelEvaluate[ClearAll[testwm]];


From this example it's clear that DistributeDefinitons is not working properly, since ParallelMap offered no speed-up. It does not help even if I add an explicit call to DistributeDefinitons. If I compile the function on each kernel, then I get the expected speed increase.

Is this (another) bug or am I doing something wrong?

I'm using mma 8.0.1 on win7 64 bit.

PS: for non compiled functions everything works as expected:

testnc[x_, n_] :=
Module[ {sum, inc}, sum = 1.0; inc = 1.0;
Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum];
Map[(testnc[1.5, 10000] + $KernelID) &, Range[2$ProcessorCount]] // AbsoluteTiming
(*
==> {0.5660324, {4.48169, 4.48169, 4.48169, 4.48169, 4.48169,
4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169, 4.48169}}
*)
ParallelMap[(testnc[1.5, 10000] + $KernelID) &, Range[2$ProcessorCount]] // AbsoluteTiming
(*
==> {0.1210069, {10.4817, 10.4817, 9.48169, 9.48169, 8.48169,
8.48169, 7.48169, 7.48169, 6.48169, 6.48169, 5.48169, 5.48169}}
*)

-
I can't reproduce your timing difference. The very first time you run ParallelMap, there is a time lag to launch the kernels. After that, I get the speed-up from ParallelMap of the kind that I would expect (roughly 5-fold on 6 cores). I am on Win 7 64 bit M8.0. – Leonid Shifrin Jun 1 '12 at 16:56
if I try DistributeDefinitions[testwm];ParallelTable[{$KernelID, OwnValues[testwm]}, {i, 1, 4}], it appears that the definitions do get distributed – acl Jun 1 '12 at 17:03 @acl Not on my machine. I ran this on a fresh kernel. Do you have 8.0.1? – Ajasja Jun 1 '12 at 17:11 8.0.1, yes. I have the same timing as you do, I was simply saying that if you do what I say, it does appear that the slave kernels know about the functions. Also t = Compile[{},$KernelID]; ParallelTable[t[], {i, 1, 4}] – acl Jun 1 '12 at 17:20
I can confirm your findings on 8.0.4. DistributeDefinitions doesn't seem to like CompiledFunctions: after compiling, try DistributeDefinitions[testwm]; ParallelEvaluate@HoldForm[testwm // Evaluate] and you'll see the slave kernels still don't know about testwm. In contrast, With[{testwmcopy = testwm}, ParallelEvaluate[testwm = testwmcopy]]; works. Don't have time to look into this now but I suspect a bug in DistributeDefinitions (it wouldn't be the first...). FWIW behaviour in version 7 is as it should be. – Oleksandr R. Jun 1 '12 at 17:31
show 1 more comment

Here's a quick workaround, so that one does not have to manually track which CompiledFunctions to distribute.

ClearAll[CompiledFunctionNames];
CompiledFunctionNames[pattern_] :=
Select[Names["Global*"],
MatchQ[Evaluate[Symbol@#], _CompiledFunction] &];

ClearAll[DistributeCompiledFunction]
DistributeCompiledFunction[name_] :=
With[{copy = Symbol[name]},
ParallelEvaluate[
If[ MatchQ[Evaluate[Symbol@name], _Symbol],(*Print["Setting ",
name];*)Evaluate[Symbol[name]] = copy]]];

ClearAll[DistributeCompiledFunctions];
DistributeCompiledFunctions[pattern_] :=
DistributeCompiledFunction /@ CompiledFunctionNames[pattern];


and some rudimentary tests:

testwm = Compile[{{x, _Real}, {n, _Integer}},
Module[{sum, inc}, sum = 1.0; inc = 1.0;
Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum],
CompilationTarget -> "WVM"];
testwm1 =
Compile[{{x, _Real}, {n, _Integer}},
Module[{sum, inc}, sum = 1.0; inc = 1.0;
Do[inc = inc*x/i; sum = sum + inc, {i, n}]; sum],
CompilationTarget -> "WVM"];
a = 5; (*a non compiled symbol -- will not get destributed*)
(*see which compiled functions are in the current context*)
CompiledFunctionNames["Global*"]
DistributeCompiledFunctions["Global*"];
(*distributing twice does not override previous definition*)
DistributeCompiledFunctions["Global*"];
(*check if it worked*)
DeleteDuplicates@ParallelEvaluate@HoldForm[testwm // Evaluate]
DeleteDuplicates@ParallelEvaluate@HoldForm[testwm1 // Evaluate]
DeleteDuplicates@ParallelEvaluate@HoldForm[a // Evaluate]
ParallelMap[(testwm[1.5, 2000000] + $KernelID) &, Range[2$ProcessorCount],
DistributedContexts -> None] // AbsoluteTiming
ParallelEvaluate[ClearAll[testwm, testwm1]];


Disclaimer: I have not yet tested this extensively.

-