For a parallel application I need to make a large list of numbers available to the parallel kernels, which I do using DistributeDefinitions. I have noticed that this is much slower when the numbers are arbitrary precision numbers compared to normal floats.
Suppose I have 2 tables:
Then Distributing the definitions (over 4 subkernels) of each
DistributeDefinitions[tab1]; // AbsoluteTiming DistributeDefinitions[tab2]; // AbsoluteTiming
completes almost instantly (0.007 seconds) for tab1 and takes over 4 seconds.
Now, of course, arbitrary precision numbers take more memory. (The ByteCount of tab2 is ten times as large as that of tab1). But even accounting for that by considering a smaller table
The time to distribute this ten times smaller table which has the same ByteCount as tab1 is still 0.4 seconds.
Can somebody explain to me why this is case? (And hopefully give some tips on avoiding this?)
PS. Related to this, why is the time taken by DistributeDefinitions approximately linear in the number of kernels?