13
$\begingroup$

I am new to parallel computing in Mathematica. I am using

LaunchKernels[16]
ParallelTable[]

to parallelize calculations. The ParallelTable consists of approx 10000 elements that are calculated using a routine which in certain cases can take a couple of minutes to finish. After starting the notebook I get after a while a bunch of errors:

LinkObject::linkd: Unable to communicate with closed link LinkObject[/path/math -subkernel -noinit -mathlink, 102, 16].

KernelObject::rdead: Subkernel connected through KernelObject[16, local] appears dead.

Parallel`Developer`QueueRun::req: Requeueing evaluations {11} assigned to KernelObject[16, local, <defunct>].

LaunchKernels::clone: Kernel KernelObject[16, local, <defunct>] resurrected as KernelObject[16, local].

Now I was wondering, what might be the cause of these errors? What would be the proper solution in this case? I have read that there is a Mathlink timeout of 15 seconds. Would that mean that if my computations take longer than 15 seconds the kernel appears dead?

$\endgroup$
6
  • 2
    $\begingroup$ If I had to guess: You're running out of memory on the kernels? $\endgroup$
    – Ajasja
    Nov 7, 2014 at 19:23
  • $\begingroup$ I have also had this problem. And frequently - generally with Integrate problems, where Integrate is sent off to various kernels to chug away at different components of a multi-part problem. I have 32G RAM and usually run 6 or 12 kernels. I don't think it is a memory problem, unless perhaps there is a memory leak, but even then I think this is most unlikely given the nature of the problems I have encountered this with. $\endgroup$
    – wolfies
    Nov 8, 2014 at 4:21
  • $\begingroup$ @wolfies: good to know that I am not alone^^ Do you remember what you did in those cases? $\endgroup$
    – ftiaronsem
    Nov 8, 2014 at 4:52
  • 1
    $\begingroup$ I had the same errors here using ParallelMap. You can try the option Method -> "FinestGrained" to minimize the time each kernel is busy without giving any feedback. $\endgroup$
    – Karsten7
    Dec 31, 2014 at 15:17
  • 1
    $\begingroup$ I have the same problem, some lengthy calculation and then parallel kernels die. RAM is, however, far from being overloaded, (still 20 GB of 24 GB) free. Setting the method to "FinestGrained" didn't help either $\endgroup$
    – Quit007
    Apr 8, 2018 at 17:07

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.