3
$\begingroup$

Consider this code:

BeginPackage["Test`"];
f1::usage = "";
f2::usage = "";
f3::usage = "";
Begin["`Private`"];
fQ[myf_] := Head[myf] === f1;
f1[n_][t_] := 0;
f2[myf1_?fQ, myf2_?fQ][t_] := 0;
f3[myf1_] := myf1[0.];
End[];
EndPackage[];

The expression evaluate on main kernel

f3[f2[f1[1], f1[1]]]
(*0*)

but when I evaluate on subkernels, it seems that it is not evaluated

LaunchKernels[];
DistributeDefinitions[f1, f2, f3];

ParallelEvaluate[Block[{data}, data = f3[f2[f1[1], f1[1]]];
  Print[data];], (Kernels[])[[1]]]

(* (kernel 1) Test`f2[Test`f1[1],Test`f1[1]][0.]*)

Where do I do wrong?

And if I distribute the definition of the fQ, it then get evaluated:

DistributeDefinitions[f1, f2, f3, Test`Private`fQ];
ParallelEvaluate[Block[{data}, data = f3[f2[f1[1], f1[1]]];
  Print[data]], (Kernels[])[[1]]]
(*  (kernel 1) 0 *)

How should I understand this behavior?

$\endgroup$
  • 1
    $\begingroup$ Somebody more knowledgeable than me will probably have a better answer, but it looks as though the parameter testing for f2 needs to call fQ, and that function definition isn't automatically distributed to the other kernels. $\endgroup$ – image_doctor Jan 31 '15 at 10:34
4
$\begingroup$

As @image_doctor notes in a comment, the problem occurs because the definition of Test`Private`fQ is not being distributed to the other kernels.

DistributeDefinitions uses Language`ExtendedFullDefinition to determine which definitions to transmit. We can see this by evaluating the following expression:

On[Language`ExtendedFullDefinition]
DistributeDefinitions[f1, f2, f3];
Off[]

(* ...
   Language`ExtendedFullDefinition::trace :
   Language`ExtendedFullDefinition[{f1,f2,f3},ExcludedContexts->{Algebra,...}] -->
   Language`DefinitionList[
      f1->{...,SubValues->{HoldPattern[f1[<<1>>][<<1>>]]:>0},...},
      f2->{...,SubValues->{HoldPattern[f2[<<2>>][<<1>>]]:>0},...},
      f3->{...,DownValues->{HoldPattern[f3[<<1>>]]:>Test`Private`myf1[0.]},...}]. >>
*)

Test`Private`fQ is absent from the generated Language`DefinitionList.

Language`ExtendedFullDefinition is normally able to trace all dependencies in a function:

ff[x_] := gg[x]
gg[x_] := x + 1
Language`ExtendedFullDefinition[ff]

(*
   Language`DefinitionList[
     ff->{...,DownValues->{HoldPattern[ff[x_]]:>gg[x]},...},
     gg->{...,DownValues->{HoldPattern[gg[x_]]:>x+1},...}
   ]
*)

But apparently it does not account for pattern tests in definitions:

ff2[x_?gg2] := 1
gg2[x_] := True
Language`ExtendedFullDefinition[ff2]

(*
   Language`DefinitionList[
     ff2->{...,DownValues->{HoldPattern[ff2[x_?gg2]]:>1},NValues->{},...}
   ]
*)

The pattern test function gg2 is missed by the dependency scan. This would appear to be a bug, but I might just be unaware of some compelling reason to ignore pattern tests in these circumstances.

As noted in the question, the work-around is to manually identify and distribute any missing dependencies. The "distribute" part of that is easy -- it is the "identify" part that can be difficult.

$\endgroup$
  • $\begingroup$ Thanks for the answer. It appears to me like a bug too. Otherwise, one have to manually distribute all the private definitions. I have reported as a bug to technical support. $\endgroup$ – xslittlegrass Jan 31 '15 at 16:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.