8
$\begingroup$

I am new to parallel computing in Mathematica. I am using

LaunchKernels[16]
ParallelTable[]

to parallelize calculations. The ParallelTable consists of approx 10000 elements that are calculated using a routine which in certain cases can take a couple of minutes to finish. After starting the notebook I get after a while a bunch of errors:

LinkObject::linkd: Unable to communicate with closed link LinkObject[/path/math -subkernel -noinit -mathlink, 102, 16].

KernelObject::rdead: Subkernel connected through KernelObject[16, local] appears dead.

Parallel`Developer`QueueRun::req: Requeueing evaluations {11} assigned to KernelObject[16, local, <defunct>].

LaunchKernels::clone: Kernel KernelObject[16, local, <defunct>] resurrected as KernelObject[16, local].

Now I was wondering, what might be the cause of these errors? What would be the proper solution in this case? I have read that there is a Mathlink timeout of 15 seconds. Would that mean that if my computations take longer than 15 seconds the kernel appears dead?

$\endgroup$
  • 1
    $\begingroup$ If I had to guess: You're running out of memory on the kernels? $\endgroup$ – Ajasja Nov 7 '14 at 19:23
  • $\begingroup$ I have also had this problem. And frequently - generally with Integrate problems, where Integrate is sent off to various kernels to chug away at different components of a multi-part problem. I have 32G RAM and usually run 6 or 12 kernels. I don't think it is a memory problem, unless perhaps there is a memory leak, but even then I think this is most unlikely given the nature of the problems I have encountered this with. $\endgroup$ – wolfies Nov 8 '14 at 4:21
  • $\begingroup$ @wolfies: good to know that I am not alone^^ Do you remember what you did in those cases? $\endgroup$ – ftiaronsem Nov 8 '14 at 4:52
  • 1
    $\begingroup$ I had the same errors here using ParallelMap. You can try the option Method -> "FinestGrained" to minimize the time each kernel is busy without giving any feedback. $\endgroup$ – Karsten 7. Dec 31 '14 at 15:17
  • 1
    $\begingroup$ I have the same problem, some lengthy calculation and then parallel kernels die. RAM is, however, far from being overloaded, (still 20 GB of 24 GB) free. Setting the method to "FinestGrained" didn't help either $\endgroup$ – Quit007 Apr 8 '18 at 17:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.