# Bug in parallel computing

The following is a simplified example. We construct a library function f and another library function g, in which the library function f is called.

f = Compile[x, Integrate[1 + t^2, {t, 0, x}], CompilationTarget -> "C"];
g = With[{f = f}, Compile[x, x + f[x], CompilationTarget -> "C"]];


CompilePrint shows the following for the function g:

1 R1 = LibraryFunction[<>, compiledFunction0, {{Real, 0, Constant}}, Real][ R0]]
2 R2 = R0 + R1
3 Return

Now we test the functions f and g on the main kernel and on two subkernels:

LaunchKernels[];
DistributeDefinitions[f,g];
f[2.3]
g[1.2]
ParallelEvaluate[f[2.3], 1]
ParallelEvaluate[g[1.2], 1]
ParallelEvaluate[g[1.2], 2]

(* Out[7]= 6.35567
Out[8]= 2.976
Out[9]= 6.35567
Out[10]= 2.976
During evaluation of In[5]:= KernelObject::rdead: Subkernel connected through KernelObject[2,local] appears dead. >>
Out[11]= \$Failed  *)


The evaluations on the main kernel and on subkernel 1 are fast. On subkernel 1 we first evaluated f and next g. On subkernel 2 we immediately called g, without having used f before. The subkernel crashes. I am running Mathematica 10.0.2 32 bits on Windows 7.

• Have you tried CompilePrint[f]? The presence of Integrate means it just calls MainEvaluate. Your simplified example may well need changing... – dr.blochwave Dec 17 '14 at 8:11
• @blochwave. Indeed this function f calls MainEvaluate. In my real situation there is no call to MainEvaluate and nevertheless the subkernel crashes. When I played a little bit with the simplified example, I found an even more amazing behaviour: when you first evaluate f and then call g on a subkernel, there is no problem; when you call g before having called f the subkernel crashes. I adapted my question accordingly. – Fred Simons Dec 17 '14 at 8:51
• Rather than using With, can you try CompilationOptions -> {"InlineCompiledFunctions" -> True} for your function g? – dr.blochwave Dec 17 '14 at 8:59
• @blochwave. In this example, yes. The result is then a function g that does not call anything else, and that works fine. In my real situation f is a library function obtained with LibraryFunctionLoad, and with such a function this workaround does not work. The bug that I report here is the fact that a subkernel crashes in a situation that it should not crash. I did not yet test if in my real situtation the crash does not happen when I first evaluate all functions that will be called on all subkernels. Anyway, I think that this should be superfluous. – Fred Simons Dec 17 '14 at 9:18
• I have the same thing going on - Mathematica 10.4, Windows 10. Whenever I call Parallel table with any lengthy amount of time it crashes and gives these same types of warnings. – Brent Jul 20 '16 at 13:25

As demonstrated by the asker, Mathematica is failing to distribute the definitions of C-compiled library functions in a way that can be called by other C-compiled functions.

I agree this is a bug. I have submitted a bug report to Wolfram Research and hope to update the post to include the result of that.

1. One workaround is to compile to C only in the last function. You can use

CompilationOptions->{"InlineCompiledFunctions"->True}

to ensure it all goes to optimized C-code. (if you remove the CompilationTarget->"C" it compiles to the Wolfram Virtual Machine.)

2. Another option is to first call all of the child functions from the parallel Kernels, which appear to get loaded properly for use. For your example

LaunchKernels[];
f = Compile[x, Integrate[1 + t^2, {t, 0, x}], CompilationTarget -> "C"];
g = With[{f = f}, Compile[x, x + f[x], CompilationTarget -> "C"]];
ParallelEvaluate[f];
ParallelEvaluate[g[1.2], 2]


{2.976, 2.976}

Although a question wasn't asked, I hope one this solves the askers problems, as it solved mine.

Update 7/21/2016: I received an email containing the following.

I was able to reproduce the issue, and consequently, I filed a report with our development team raising the issues and also shared your contact information with them so you can be notified once the issue is resolved.

Update 8/26/2016: The situation appears the same in version 11.0.0.0.

• +1. Note ParallelEvaluate[f] is sufficient in your second option (evaluate only f instead of f[2,3]). – Michael E2 Jul 20 '16 at 20:41
• Nice - I edited accordingly. – Brent Jul 20 '16 at 20:54
• "Mathematica is failing to distribute the definitions of C-compiled library functions" <- I am not sure if this can be called a bug, although it is definitely not user-friendly. A LibraryFunction is essentially a reference to an external resource. To make it work properly, it should be created/loaded separately on each subkernel. There are many such examples where something created on one kernel can't be used on another. Think opening a file. You can an InputStream expression. It won't work on another kernel unless you open the file there too. – Szabolcs Jul 20 '16 at 21:07
• Or think loading a package. You need to use ParallelNeeds and can't just distribute. Or think using managed library expressions. Do << TriangleLink , then TriangleCreate[]. You'll get a TriangleExpression`. It can be distributed as an expression, but on subkernels it simply loses its meaning (which is a reference to some C-side data structure present only in main kernel memory). There will naturally always be such cases, and it is unreasonable to call them bugs. Perhaps compiled functions are a bit special because they are so common. But the reasonable thing is still to ... – Szabolcs Jul 20 '16 at 21:09