5
$\begingroup$

So I was trying a very simple example (Mathematica 11.0.1)

AbsoluteTiming[
 A1 = Table[
    RandomReal[WorkingPrecision -> 30], {ii, 1, 1000}, {jj, 1, 
     1000}];]
{0.752044, Null}

And then in parallel

AbsoluteTiming[
 A1 = ParallelTable[
    RandomReal[WorkingPrecision -> 30], {ii, 1, 1000}, {jj, 1, 
     1000}];]
{6.35024, Null}

(which is after running it at least once so that definitions and stuff get distributed). It is clearly much slower than Table. This timing is independent of the number of kernels I launch - that is - it remains unchanged for LaunchKernels[n], where n is any integer different than 0 up to the maximum number of kernels I have (tried it on a machine with 12).

I also tried doing thing like

$MinPrecision = 30; $MaxPrecision = Infinity;
ParallelEvaluate[$MinPrecision = 30; $MaxPrecision = 
  Infinity;]; DistributeDefinitions[$MinPrecision, $MaxPrecision];

and then

AbsoluteTiming[
 A1 = ParallelTable[
    RandomReal[WorkingPrecision -> $MinPrecision], {ii, 1, 1000}, {jj,
      1, 1000}];]
{6.38118, Null}

Which is again much slower than Table. I would like to know why is this happening, so that I can avoid this type of behaviour.

$\endgroup$
  • 1
    $\begingroup$ Have you seen this? $\endgroup$ – J. M. is away Mar 31 '17 at 10:34
  • 1
    $\begingroup$ What do you mean by "doesn't work" ? Which Mathematica version? For me in 11.1 the generated random numbers have precision 30. If you are surprised by the timing: say so. $\endgroup$ – Rolf Mertig Mar 31 '17 at 10:47
  • $\begingroup$ @RolfMertig I changed the title (and some lines) and added Mathematica's version to make it clearer. I am frustrated witht he timing - yes. $\endgroup$ – ThunderBiggi Mar 31 '17 at 11:21
  • $\begingroup$ @J.M. I looked at it and as far as I understand it, Mathematica generates a different seed for each subkernel and this goes through the master kernel, which might explain why is it so slow in parallel. I tried to define a seed and destribute it to the subkernels with DistributeDefinitions, but that didn't work. $\endgroup$ – ThunderBiggi Mar 31 '17 at 11:24
  • 1
    $\begingroup$ Related: (2886), (20713), (28896),(31560), (33610), $\endgroup$ – Mr.Wizard Mar 31 '17 at 16:38
6
$\begingroup$

First of all, the best way to do this calculation is

RandomReal[1, {1000, 1000}, 
   WorkingPrecision -> 30]; // AbsoluteTiming
(* {0.312959, Null} *)

Why is the parallel version slow? There can be many reasons, and unfortunately I do not have the time to verify which one it is here. However, I strongly suspect that it is data transfer overhead.

This computation is generally very fast, but it produces huge data that needs to be transferred back to the main kernel. Moreover, that data is arbitrary precision, which means that each number needs to be sent over MathLink independently (instead of sending a packed array in one go). This is exactly the situation where I would expect the parallel tools to perform badly.

You can test this yourself by trying to pass data of this size and type through MathLink (see e.g. LinkWrite, or otherwise use DistributeDefinitions).


Update: Here's a benchmark for MathLink transfer of such a large arbitrary precision array. This shows that it is indeed the data transfer that is slow.

link = LinkCreate[LinkMode -> Loopback];

arr = RandomReal[1, {1000, 1000}, WorkingPrecision -> 30];

LinkWrite[link, arr] // AbsoluteTiming
(* {4.30156, Null} *)

arr2 = LinkRead[link]; // AbsoluteTiming
(* {6.9221, Null} *)

The slowness is due to the fact that this is not a packed array (arbitrary precision arrays cannot be), and thus each element is sent with a separate MathLink call.

Compare how much faster it is to transfer a packed array:

parr = RandomReal[1, {1000, 1000}]; (* packed machine precision array *)

LinkWrite[link, parr] // AbsoluteTiming
(* {0.004155, Null} *)

parr2 = LinkRead[link]; // AbsoluteTiming
(* {0.005705, Null} *)
$\endgroup$
  • $\begingroup$ I have marked this question as a duplicate. Please review and let me know if you disagree. If you do not this answer might be moved to one of those older questions, or added to an existing answer. If a merge is desired please also let me know. $\endgroup$ – Mr.Wizard Mar 31 '17 at 16:40
  • 1
    $\begingroup$ @Mr.Wizard I think I do disagree. There are several things that can slow down parallel calculations, and the other threads seems to show different problems. The fact that the numbers are arbitrary precision is very important here. Try my benchmark after removing WorkingPrecision -> 30. The transfer time drops to 0.004 seconds (!!). And ParallelTable is no longer slower than Table (though it isn't faster either on my computer, and it is still not the best way to generate a random matrix). $\endgroup$ – Szabolcs Mar 31 '17 at 16:46
  • $\begingroup$ Alright. Question reopened. Your answer already has my vote. $\endgroup$ – Mr.Wizard Mar 31 '17 at 16:48
  • $\begingroup$ I had checked that with machine precision ParallelTable is as fast as Table, but I didn't want to make the question longer. I also didn't imagine that all the slowdown is due to the increased precision (even though I put it in the title, as I knew it is different from what has been done in other threads) - I thought that I am missing something simple and after J.M. gave me links me to the subtleties with SeedRandom, I thought that the problem might be there. I am glad the question was reopened and I hope that others will benefit. Thank for the help and doing the thing with Link, Szabolcs $\endgroup$ – ThunderBiggi Mar 31 '17 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.