I need to generate a lot of point according to a given distribution. I don't have an analytic form of this distribution so I am using an InterpolatingFunction of ~100 points computed numerically. I then normalize this interpolation and I define a ProbabilityDistribution from which I generate numbers using RandomVariate.
First of all this leads to a really slow evaluation of RandomVariate (compared to cases in which the ProbabilityDistribution is known analitically), but that's not the main point of this question (it might be related to the fact that my pdf is exponentially decreasing and defined on a quite wide interval and so is close to zero most of the time... maybe?).
Anyway, I use these points for some heavy computations and so I try to parallelize as much as possible. This lead me to the nice discovery that RandomVariate takes twice as long when done in parallel. Here is a short example code that reproduces what I want to say:
(* I create a random pdf just as an example *)
pdf = Table[{x, RandomReal[]}, {x, 1, 10, 0.5}];
intpdf = Interpolation@pdf;
norm = Integrate[intpdf[x], {x, 1, 10}];
dist = ProbabilityDistribution[intpdf[x]/norm, {x, 1, 10}];
(* no parallelization *)
AbsoluteTiming[RandomVariate[dist, 1000000]][[1]]
(* Out = 7.28052 *)
(* parallelization on 8 subkernels *)
ParallelTable[
AbsoluteTiming[RandomVariate[dist, 1000000]][[1]], {i, 1, 8}]
(* Out = {13.021, 12.7993, 13.8139, 13.417, 12.8863, 13.0874, 13.3788, 13.7836} *)
As you can see, the evaluation of the same operation takes approximately twice as long when done in the subkernels. I have read that this is due to poor parallelization properties of interpolating functions, but I can't find a way to fix this and have my code run fast. What can I do?
SmoothKernelDistributionachieve what you need. See reference.wolfram.com/language/guide/…RandomVariateruns at least two orders of magnitude faster with them. $\endgroup$ – Theo Tiger May 27 at 20:28