6
$\begingroup$

If a condition is met within a sub-kernel in the function being mapped, how does one cancel the whole ParallelMap operation?

ParallelMap[
 Check[Pause[1]; 
   If[# > 10, Print[$KernelID, " failed on ", #]; $Failed, 
    Print[$KernelID, " suceeded on ", #]], AbortKernels[]] &, 
 Range[20]]

This seems like something commonplace, but I can't remember how or find it anywhere in the docs!

$\endgroup$
  • $\begingroup$ Looks like you want ParallelTry $\endgroup$ – Lukas Lang Mar 8 at 12:31
  • 1
    $\begingroup$ It’s sort of the opposite - ParallelTry returns the first successful result, I want to stop on the first failure $\endgroup$ – M.R. Mar 8 at 12:37
2
$\begingroup$

Adapting the code from ParallelTry to do what you need, we get

ComputeUntilFailure[f_, _[args___]] := Module[
  {ids, id, r, res = <||>},
  DistributeDefinitions[{f, args}];
  ids = List @@ ParallelSubmit /@ (# -> f@# &) /@ Hold[args];
  While[Length@ids > 0,
   {r, id, ids} = WaitNext[ids];
   If[FailureQ@Last@r,
    AbortKernels[];
    Clear@ids;
    Break[],
    AppendTo[res, r]
    ]
   ];
  res
  ]

Basically, this uses ParallelSubmit to launch the computations on parallel kernels. The results are then processed one by one on the main kernel using WaitNext, until one of them is a failure, at which point we abort the remaining computations.

Applying this to the example given in the question:

ComputeUntilFailure[
  (
    Pause@1; 
    If[# > 10,
      Print[$KernelID, " failed on ", #];
      $Failed,
      Print[$KernelID, " succeeded on ", #]
    ]
  ) &,
  Range[20]
]
(* 6 succeeded on 1 *)
(* 5 succeeded on 2 *)
(* 4 succeeded on 3 *)
(* 3 succeeded on 4 *)
(* 2 succeeded on 5 *)
(* 1 succeeded on 6 *)
(* 6 succeeded on 7 *)
(* 5 succeeded on 8 *)
(* 4 succeeded on 9 *)
(* 3 succeeded on 10 *)
(* 2 failed on 11 *)
(* 1 failed on 12 *)
(* <|1 -> Null, 2 -> Null, 3 -> Null, 4 -> Null, 5 -> Null, 
 6 -> Null, 7 -> Null, 8 -> Null, 9 -> Null, 10 -> Null|> *)
$\endgroup$
  • $\begingroup$ Thanks, can you give an example usage of this? $\endgroup$ – M.R. Mar 8 at 22:17
  • $\begingroup$ I'm wondering why does it not abort after it fails on 11? $\endgroup$ – M.R. Mar 12 at 15:52
  • $\begingroup$ @M.R. The issue is that all the tasks finish almost exactly simultaneously. So when the main thread got the first failed result from Kernel 2, Kernel 1 also produces a failed result (almost) the same instant, before the main kernel has any chance to abort the computation. If you replace the Pause[1] statement with Pause@RandomReal[] (to force the different kernels to be less synchronous), it will only generate a single failed result in most cases. $\endgroup$ – Lukas Lang Mar 12 at 16:51
  • $\begingroup$ Interesting, I’d consider that a bug in the parallel functions $\endgroup$ – M.R. Mar 12 at 17:45
  • $\begingroup$ @M.R. why? I would say that that's just the nature of concurrent programs (If you want to avoid that things finish simultaneously, you can simply remove the parellization ;-) ) Also, note that the returned list contains neither of the two failed instances, as expected $\endgroup$ – Lukas Lang Mar 12 at 18:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.