2
$\begingroup$

For a parallel application I need to make a large list of numbers available to the parallel kernels, which I do using DistributeDefinitions. I have noticed that this is much slower when the numbers are arbitrary precision numbers compared to normal floats.

Suppose I have 2 tables:

tab1=Table[RandomReal[],{10^6}]
tab2=SetPrecision[tab1,10]

Then Distributing the definitions (over 4 subkernels) of each

DistributeDefinitions[tab1]; // AbsoluteTiming
DistributeDefinitions[tab2]; // AbsoluteTiming

completes almost instantly (0.007 seconds) for tab1 and takes over 4 seconds.

Now, of course, arbitrary precision numbers take more memory. (The ByteCount of tab2 is ten times as large as that of tab1). But even accounting for that by considering a smaller table

tab3=tab2[[1;;-1;;10]];

The time to distribute this ten times smaller table which has the same ByteCount as tab1 is still 0.4 seconds.

Can somebody explain to me why this is case? (And hopefully give some tips on avoiding this?)

PS. Related to this, why is the time taken by DistributeDefinitions approximately linear in the number of kernels?

$\endgroup$
  • 2
    $\begingroup$ I suspect that the real question is about the performance of MathLink. It is entirely possible that I remember wrong (!!), but I think that arbitrary precision numbers are transferred as strings while machine precision numbers as normal binary floating point numbers. Also, the C MathLink API has functions for putting/getting a whole array of machine numbers, but not arbitrary precision numbers. $\endgroup$ – Szabolcs Oct 28 '16 at 14:21
  • 1
    $\begingroup$ In the end it is not the ByteCount that matters. ByteCount tells how how much space something would take up in kernel memory without any subexpression sharing (see Share). I'm pretty sure that expressions don't use the same representation as in kernel memory when they are being transferred through a MathLink link. There are also some weird effect relating to the size of the encoding, see e.g. how transfer through a JSON string was faster than standard MathLink array transfer here. $\endgroup$ – Szabolcs Oct 28 '16 at 14:26
  • $\begingroup$ The fact that the distributed array is packed or not packed seems to have an impact (tab1 is packed but not tab2). Compare for instance tab1u = Developer`FromPackedArray[tab1]; RepeatedTiming[DistributeDefinitions[tab1u];] and RepeatedTiming[DistributeDefinitions[tab3];]. $\endgroup$ – user31159 Oct 28 '16 at 14:26
  • 1
    $\begingroup$ "why is the time taken by DistributeDefinitions approximately linear in the number of kernels?" I assume it's because the main kernel has to send separately to each kernel. At least the MathLink put operations must be performed as many times as you have subkernels. Since they're all done by the main kernel, they are likely done sequentially. This is an educated guess, as I don't fully understand MathLink, neither is it fully documented. $\endgroup$ – Szabolcs Oct 28 '16 at 14:51
  • 1
    $\begingroup$ To add (I hope) to what @Szabolcs and others have said, with bignums there is a conversion to/from formats whereas with a packed array of machine doubles the transfer over MathLink requires no format conversions. I would expect these conversions to be costly in terms of speed. $\endgroup$ – Daniel Lichtblau Oct 28 '16 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.