I have used code from Tell ParallelMap[] to use just specific kernels :
chunkenize[data_, nkernels_] :=
Partition[data, Quotient[Length[data], nkernels]]
MyParallelMap[f_, data_, kernels_] :=
Module[{chunks = chunkenize[data, Length[kernels]]},
Block[{subdata},
MapIndexed[ParallelEvaluate[subdata = #1, kernels[[First[#2]]]] &,
chunks];
DistributeDefinitions[f];
ParallelEvaluate[Map[f, subdata], kernels]]]
with
m = 5 10^3;
f[y_] := (3 y^3)/Sqrt[y^2 + 1] // N[#, 10^5] &
kernels = LaunchKernels[]
to get
{"KernelObject"[1, "local"], "KernelObject"[2, "local"],
"KernelObject"[3, "local"], "KernelObject"[4, "local"],
"KernelObject"[5, "local"], "KernelObject"[6, "local"]}
Then
ClearSystemCache[];
MyParallelMap[f, Range[m], kernels]; // AbsoluteTiming
yields
{100.934, Null}
whereas the non-parallel version
ClearSystemCache[];
Range[m] // f; // AbsoluteTiming
is almost 5 times faster:
{21.9527, Null}
If I do it with N[#, 10^4] & instead of N[#, 10^5] &, then the parallel calculation is "only" about 2 times slower. If I do it for N[Log[#], 10^4] & instead of f[], the parallel calculation is about 2--3 times faster -- something to be expected, I think. What could make f[] so drastically different from Log[]?
More importantly, why would the parallel calculation be so much slower than the non-parallel one in any case? Any fix to this? Thank you.
Log
and it took ages (175 sec. with all kernels running, vs 45 sec. another time with kernels running in sequence). I don't understand theLog
result at all. There are a few others on this site who understand parallel processing much better than I do. I hope they'll see your question and be able to answer it. $\endgroup$