14
$\begingroup$

Under Mathematica 8 (Linux x84_64) I was able to run an external program in parallel from a Mathematica package. However this procedure breaks down under Mathematica 9.

Within my Mathematica package I created some artificial links for the sub-kernels to be synchronized with the links created by the master kernel. Otherwise the link numbering of the sub-kernels does not coincide with the numbering created by the master kernel. This procedure is different from that suggested by

How to run mathlink external commands in parallel?

Here is the relevant code I used in my Mathematica package that did the job successfully under Mathematica 8.

Which[SameQ[Global`$ParaMode,"False"] && SameQ[Global`$NotebookMode,"False"],False,
      SameQ[Global`$ParaMode,"False"] && SameQ[Global`$NotebookMode,"True"], False,
      SameQ[Global`$ParaMode,"True"]  && SameQ[Global`$NotebookMode,"False"],kc=$KernelCount;Table[LinkCreate[],{kc-1}],
      SameQ[Global`$ParaMode,"True"]  && SameQ[Global`$NotebookMode,"True"],kc=$KernelCount;Table[LinkCreate[],{kc+6}]
      ];

Now let us see what will happen under Mathematica 9. To keep the example small I launch only two sub-kernels.

In[1]:= LaunchKernels[2];

In[2]:= $KernelCount

Out[2]= 2

After loading my package into the master kernel and sub-kernels I have the following link connections for the master kernel

In[50]:= Links[]


Out[50]= {LinkObject[/pfs/data/software/all/mathematica/9.0/Executables/math\

>      -subkernel -noinit -mathlink, 83, 1], 

>    LinkObject[/pfs/data/software/all/mathematica/9.0/Executables/math -subkernel\

>      -noinit -mathlink, 84, 2], LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddma\

>      thlink, 123, 3], LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2g\

>      mp, 124, 4], LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2, 125, 

>     5]}

and here the link connections for the two sub-kernels.

In[51]:= ParallelEvaluate[Links[]]


Out[51]= {{LinkObject[q63_shm, 3, 1], 

>     LinkObject[[email protected],[email protected], 101, 2], 

>     LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink, 102, 3], 

>     LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 103, 4], 

>     LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2, 104, 5]}, 

>    {LinkObject[35c_shm, 3, 1], 
LinkObject[[email protected],[email protected], 

>      99, 2], LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink, 100, 3], 

>     LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 101, 4], 

>     LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2, 102, 5]}}

On a first glance everything seems fine, since the links between the master kernel and its sub-kernels were synchronized as intended.

Now, I want to call a function from my Mathematica package in parallel that makes use of the external program.

In[54]:= 
 a2a=ParallelTable[StrongEpsCore3dV6[ExpGame11,EpsStrValues -> t, ViewKernelSol -> True, KernelCoord -> ker, ViewNucleolusSol -
> True, NucleolusCoord -> mnuc, ViewShapleySol -> True, ShapleyCoord -> shv, ShowCore->True, ShowImputationSet -> True], {t, 30, detlow, -(1/40)
}]

But to my big surprise, the procedure failed. The strange thing here is that both sub-kernels trying to use the connection build up by the master kernel instead of using their own connections.

From KernelObject[2, local]:
LinkObject::linkn: 
   Argument LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 124, 4] in\

>   
    LinkWrite[LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 124, 4],\

>   
     CallPacket[1, {5, 5, {{0, 1, 0, 0, 0}, {0, 0, 1, 0, 0}, {0, 0, 0, 1, 0},
         {-25369617/<<9>>7, 0, 0, 0, 1}, {-1, 1, 1, 1, 1}}}]] has an invalid
     LinkObject number; the link may be closed.


From KernelObject[1, local]:
LinkObject::linkn: 
   Argument LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 124, 4] in\

>   
    LinkWrite[LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 124, 4],\

>   
     CallPacket[1, {5, 5, {{0, 1, 0, 0, 0}, {0, 0, 1, 0, 0}, {0, 0, 0, 1, 0},
         {-25369617/<<9>>7, 0, 0, 0, 1}, {-1, 1, 1, 1, 1}}}]] has an invalid
     LinkObject number; the link may be closed.

My question are now as follows:

How I have to modify the above code so that the sub-kernels are using their own connections rather than that of the master kernel?

How can I avoid this unintended behavior?

It might also be helpful to know what changes have been made in the MathLink protocol from Mathematica 8 to 9.

Update

In the meantime I made some small progress while starting from scratch without calling any package defined functions. Contrary to my first observation setting the option

DistributedContexts -> None

might have an effect whenever the function of the external program is called directly. To see this, let us call in a first step the external program directly without setting the option DistributedContexts -> None, then we still have the same failure

In[17]:= Parallelize[Table[AllVertices[6,5,string1],{2}]]
From KernelObject[2, local]:
LinkObject::linkn: 
   Argument LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 85, 3] in 
    LinkWrite[LinkObject[/pfs/data/home/kit/xxxx/xxxxx/bin64/cddmathlink2gmp, 85, 3], 
     CallPacket[0, {6, 5, {{0, 1, 0, 0, 0}, {0, 0, 1, 0, 0}, {0, 0, 0, 1, 0}, {0, 0,
         0, 0,<<11>>, 1, 1, 1}, {1, -1, -1, -1, -1}}}]] has an invalid LinkObject
     number; the link may be closed.
From KernelObject[1, local]:
LinkObject::linkn: 
   Argument LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 85, 3] in 
    LinkWrite[LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 85, 3], 
     CallPacket[0, {6, 5, {{0, 1, 0, 0, 0}, {0, 0, 1, 0, 0}, {0, 0, 0, 1, 0}, {0, 0,
         0, 0,<<11>>, 1, 1, 1}, {1, -1, -1, -1, -1}}}]] has an invalid LinkObject
     number; the link may be closed.

Now, calling the function directly with the option DistributedContexts -> None helps in this case, I got the desired result

 In[17]:= Parallelize[Table[AllVertices[6,5,string1],{2}],DistributedContexts -> None]
Out[17]= {{{{{1, 1,  0,  0,  0}, {1,  0, 1,  0,  0}, {1,  0,  0, 1,  0}, 

>       {1,  0,  0,  0, 1}}, {}}, {{2, 3, 4, 5, 6}, {1, 3, 4, 5, 6}, {1, 2, 4, 5, 6}, 

>      {1, 2, 3, 5, 6}}}, {{{{1, 1,  0,  0,  0}, {1,  0, 1,  0,  0}, 

>       {1,  0,  0, 1,  0}, {1,  0,  0,  0, 1}}, {}}, 

>     {{2, 3, 4, 5, 6}, {1, 3, 4, 5, 6}, {1, 2, 4, 5, 6}, {1, 2, 3, 5, 6}}}}

Nevertheless, when I try to invoke the above package function again with the option DistributedContexts -> None to call indirectly the external program, then I still get the same failure. Nothing has improved.

It seems that the option DistributedContexts -> None must be put in the right place in order to have an effect. But how I have to do that?

Update 2

To simulate a package function call of the external program, I defined ceteris paribus the following indirect function call

 In[15]:= callVert[m_Integer,d_Integer,a_String]:=AllVertices[m,d,string1];
 In[16]:= DistributeDefinitions[callVert]
 Out[16]= {callVert} 

To observe that the function is well defined, we call this function first by the master kernel and obtain

 In[19]:= callVert[6,5,string1]
 Out[19]= {{{{1, 1,  0,  0,  0}, {1,  0, 1,  0,  0}, {1,  0,  0, 1,  0}, 

>      {1,  0,  0,  0, 1}}, {}}, {{2, 3, 4, 5, 6}, {1, 3, 4, 5, 6}, {1, 2, 4, 5, 6}, 

>     {1, 2, 3, 5, 6}}}

which coincides with the above result.

However, calling now the above indirect function using Parallelize with option DistributedContexts :> None, we attain still the same failure

Parallelize[Table[callVert[6,5,string1],{2}],DistributedContexts :> None]
From KernelObject[2, local]:
LinkObject::linkn: 
   Argument LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 88, 3] in 
    LinkWrite[LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 88, 3], 
     CallPacket[0, {6, 5, {{0, 1, 0, 0, 0}, {0, 0, 1, 0, 0}, {0, 0, 0, 1, 0}, {0, 0,
         0, 0,<<11>>, 1, 1, 1}, {1, -1, -1, -1, -1}}}]] has an invalid LinkObject
     number; the link may be closed.
From KernelObject[1, local]:
LinkObject::linkn: 
   Argument LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 88, 3] in 
    LinkWrite[LinkObject[/pfs/data/home/kit/xxxx/xxxx/bin64/cddmathlink2gmp, 88, 3], 
     CallPacket[0, {6, 5, {{0, 1, 0, 0, 0}, {0, 0, 1, 0, 0}, {0, 0, 0, 1, 0}, {0, 0,
         0, 0,<<11>>, 1, 1, 1}, {1, -1, -1, -1, -1}}}]] has an invalid LinkObject
     number; the link may be closed.

Out[21]= {$Failed, $Failed}

I even tried

In[17]:= $DistributedContexts = None;
In[18]:= Map[SetOptions[#,DistributedContexts :> None]&, {Parallelize}];
In[19]:= Options[Parallelize]
Out[19]= {Method -> Automatic, DistributedContexts :> None}

without any effect.

What's going on here? Why is it not possible to apply the option DistributedContexts :> None in an indirect function call for an external program?

$\endgroup$
2
  • 1
    $\begingroup$ Can you try ParallelEvaluate/ParallelTable with option DistributedContexts -> None? I find that the automatic distribution of definitions usually causes more problems than it saves and you may have encountered an issue with it here. As for the MathLink protocol: no (externally observable) changes were made between 8 and 9. We are still on MLINTERFACE version 3. $\endgroup$ May 21, 2013 at 2:00
  • 1
    $\begingroup$ I tried your suggestions with option DistributedContexts -> None. However, without any effect. Still the same failure. $\endgroup$ May 21, 2013 at 8:12

1 Answer 1

5
$\begingroup$

I solved the problem! The solution is quite simple. It is not allowed to distribute the definition of an indirect function to the sub-kernels. Thus, if we do not use ceteris paribus

In[17]:= (* DistributeDefinitions[callVert] *)

then setting the option DistributedContexts :> None by

In[18]:= Map[SetOptions[#,DistributedContexts :> None]&, {Parallelize}];
 In[19]:= Options[Parallelize]
 Out[19]= {Method -> Automatic, DistributedContexts :> None}

we can successfully call the external program by the indirect function callVert, as we can see next

In[25]:= Parallelize[Table[callVert[6,5,string1],{2}]]
Out[25]= {{{{{1, 1,  0,  0,  0}, {1,  0, 1,  0,  0}, {1,  0,  0, 1,  0}, 

>       {1,  0,  0,  0, 1}}, {}}, {{2, 3, 4, 5, 6}, {1, 3, 4, 5, 6}, {1, 2, 4, 5, 6}, 

>      {1, 2, 3, 5, 6}}}, {{{{1, 1,  0,  0,  0}, {1,  0, 1,  0,  0}, 

>       {1,  0,  0, 1,  0}, {1,  0,  0,  0, 1}}, {}}, 

>     {{2, 3, 4, 5, 6}, {1, 3, 4, 5, 6}, {1, 2, 4, 5, 6}, {1, 2, 3, 5, 6}}}}

Referred to my Mathematica package, this means that I made the mistake to distribute the package contents besides with ParallelNeeds also with DistributeDefinitions to all sub-kernels to be sure that the complete package contents is distributed to the sub-kernels. In the past, I noticed that the function definitions were not distributed properly. This fixed the problem under Mathematica 8, but caused a break down of the external parallel computation under Mathematica 9.

Thus, if I make now a proper call of my Mathematica function and launching 12 sub-kernels

 In[55]:= AbsoluteTiming[a2a=ParallelTable[StrongEpsCore3dV6[ExpGame11,EpsStrValues -> t, ViewKernelSol -> True, KernelCoord -> ker, ViewNucleolusSol -> True, NucleolusCoord -> mnuc, ViewShapleySol -> True, ShapleyCoord -> shv, ShowCore->True, ShowImputationSet -> True], {t, 30, detlow, -(1/40)},DistributedContexts -> None]]

Out[55]= {19.135421, {-Graphics3D-, -Graphics3D-, -Graphics3D-, ... }

then the function generates 2158 graphics in parallel in about 19 seconds rather than in 80 seconds.

 In[57]:= Length[a2a]
 Out[57]= 2158 

We are done!

$\endgroup$
3
  • $\begingroup$ Thanks @OleksandrR for his useful comment. $\endgroup$ May 24, 2013 at 19:19
  • $\begingroup$ Hi Holger, if you solve your own problem, you can post that as an answer instead of adding it to the question and posting a pointer to it. For now, I've moved the relevant content from your question to this post. $\endgroup$
    – rm -rf
    May 24, 2013 at 21:19
  • $\begingroup$ Thanks for your hint, but this was not so clear from the menu. I will keep it in mind. $\endgroup$ May 25, 2013 at 6:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.