1
$\begingroup$

I have two functions, f and g, that I wish to apply to large data sets, data1 and data2, and assign them to variables X and Y. Because data1 and data2 are large, and because f and g are computationally demanding ("expensive"), I wish to do the following computations in parallel:

X = f[data1];
Y = g[data2];

or, likewise,

{X, Y} = {f[data1], g[data2]};

How do I compute X and Y in parallel in general? The answer to this question seems like it should be obvious, but it's not obvious to me.

If f and g were identical, I could use ParallelMap, for example:

{X, Y} = ParallelMap[f, {data1, data2}];

Similarly, if data1 and data2 were identical, I might be able to use ParallelTable:

{X, Y} = ParallelTable[i[data1], {i, {f, g}]

But how can I compute X and Y in parallel in general?

I have tried the following methods -- (1), (2), and (3) -- and the results seem to suggest that either I am not using Parallelize correctly or what I'm trying to accomplish cannot be done with Parallelize. In the following, I have used example functions (calculating lengths of the result of FactorInteger) in place of my own functions, f and g, since they are rather complicated. So I've borrowed the idea for a simple yet "expensive" computation using FactorInteger from this documentation page.

Print["Number of kernels before launch: ", $KernelCount];
LaunchKernels[2];
Print["Number of kernels after launch: ", $KernelCount];

(* Do two "expensive" calculations and assign the results to X and Y, respectively *)
(* (1) Do the computations in sequence *)
AbsoluteTiming[
X = AbsoluteTiming[Table[Length@FactorInteger[10^50 + i], {i, 20}]];
Y = AbsoluteTiming[Table[Length@FactorInteger[10^50 + i], {i, 20}]];
]

(* (2) Attempt in vain to do the computations in parallel using Parallelize *)
AbsoluteTiming[
Parallelize[
X = AbsoluteTiming[Table[Length@FactorInteger[10^50 + i], {i, 20}]];
Y = AbsoluteTiming[Table[Length@FactorInteger[10^50 + i], {i, 20}]];
]]

(* (3) Use Parallel Table to do the computations in parallel;
then assign the results to X and Y *)
AbsoluteTiming[
{X, Y} = ParallelTable[
AbsoluteTiming[Table[Length@FactorInteger[10^50 + i], {i, 20}]], {2}];
]

(* Method (3) speeds up the computations somewhat by performing them in parallel. *)
(* However, if the two computations are significantly different in form,
then ParallelTable cannot in general be used -- as far as I can tell. *)

Number of kernels before launch: 0

Number of kernels after launch: 2

{5.209785, Null}

{5.100558, Null}

{4.185426, Null}

$\endgroup$

1 Answer 1

2
$\begingroup$

We can use ParallelSubmit (introduced in version 7.0) and WaitAll to do what you want simply and effectively. But first a hopefully illuminating example as to how ParallelSubmit works.

Evaluation sequence of ParallelSubmit

(*job1*)ParallelSubmit[Labeled[Pause[2]; TimeObject[Now], $KernelID]]
(*job2*)ParallelSubmit[Labeled[TimeObject[Now], $KernelID]]
WaitAll@Out[{-2, -1}]

enter image description here

As we can see, job2 completed before job1 even though it normally (i.e. without ParallelSubmit) would enter the evaluation stack after job1. So WaitAll submits these tasks to be evaluated at the same time (flagged to be evaluated on different kernels), Mathematica receives the collection of tasks, sends each of them onto their own kernel, then evaluates them simultaneously. I'm thinking of it like the internals of a post office sorting facility.

General usage

So, with that in mind, we can simply wrap each evaluation in a ParallelSubmit and then send both with a WaitAll as

fd1 = ParallelSubmit@f[data1]
gd2 = ParallelSubmit@g[data2]
{X, Y} = WaitAll@{fd1, gd2}

Though, when manually evaluating custom functions on parallel/remote kernels, one should take care to make sure the other kernels know the definitions of said functions. See more on Resource Sharing in Parallel Computing.

FactorInteger example

And in the example we can an "effective" evaluation time by taking maximum of the parallel timings.

ParallelSubmit@First@AbsoluteTiming[Table[Length@FactorInteger[10^50 + i], {i, 20}];];
ParallelSubmit@First@AbsoluteTiming[Table[Length@FactorInteger[10^50 + i], {i, 20}];];
Max@WaitAll@Out[{-2, -1}]

1.21117

NOTE: beware of parallel overhead because sometimes sequential evaluation can actually be quicker than doing so in parallel.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.