(The problem has been narrowed down, and solved to a more simple one described at the end of the post beside UPDATE 3)
I'm doing some numerical work that takes quite a long time to run on my computer (which has 4 kernels). I got access to a cluster which has a Mathematica license that lets me use up to 80 kernels.
I made a few simple tests to find out how efficient it turns out to be. The first one being
ParallelTable[Pause[1]; i, {i, 1, 80}];// AbsoluteTiming
{1.171367, Null}
which tells me that all 80 kernels are running in parallel.
The next test I ran on my computer (not the cluster) was
Table[Sqrt[i], {i, 1, 10^6}]; // AbsoluteTiming
{11.3152, Null}
and
ParallelTable[Sqrt[i], {i, 1, 10^6}]; // AbsoluteTiming
{13.5914, Null}
which somehow tells me that 4 kernels in parallel run slower than 1 alone (maybe because it takes time to rejoin all information sent to the kernels?).
The same test using only 1 kernel from the cluster returns {13.661838, Null} from which I deduce that their processor is less powerful than mine. More interestingly, when I run all 80 kernels, I get {19.057895, Null}. I'm a bit disappointed with this result, but I'm sure that I'm missing a detail which explains this discrepancy.
Moreover, what I'm really interested is to create a table (using ParallelTable) of the function (for 0<=w<=1000 and 0<=b<=300; the options used give me an overall error of ~0.1% ,which is acceptable):
FAuR[w_, b_] := If[b == 0, 0, NIntegrate[1/u Sqrt[u^2 - (w/gAu)^2]
BesselJ[1, b Sqrt[u^2 - (w/gAu)^2]] FFAuR[u^2], {u, w/gAu,
40}, MinRecursion -> 3, MaxRecursion -> 100,
WorkingPrecision -> 20, PrecisionGoal -> 10,
AccuracyGoal -> Infinity] ]
where gAu=19700/183 and FFAuR is an interpolated function which can be imported using the command FFAuR = << "FF-Au.m"(the file FF_Au.m can be downloaded from the link provided at the end).
Running this function on my computer I get
FAuR[1, 1] // AbsoluteTiming
NIntegrate::slwcon: Numerical integration converging too slowly; suspect one of the following: singularity, value of the integration is 0, highly oscillatory integrand, or WorkingPrecision too small. >>
{5.26011, 0.086004396132821223043}
while running it on the cluster I get
FAuR[1, 1] // AbsoluteTiming
NIntegrate::slwcon: Numerical integration converging too slowly; suspect one of the following: singularity, value of the integration is 0, highly oscillatory integrand, or WorkingPrecision too small. >>
{431.500101, 0.086004396132821223043}
It takes 80x more time to run on the cluster, what is happening? I was really hoping to have a great improvement and being able to get the most out of the cluster but these results are telling me otherwise. Any idea?
If anyone is interested in running the code, I uploaded the FF_Au.m (32 MB) file to my Google Drive:
https://drive.google.com/open?id=0B3YRtWnRI0v2NUdSWnVOWVBUVXM
My computer is running Mathematica 11.0 on Windows 10, and the cluster is running Mathematica 10.0.
UPDATE 1 (21/08): I found out that the problem is caused by the interpolated function FFAuR. Integrating it numerically alone takes 435 seconds on the cluster and 5 seconds on my computer. However, plotting takes the same amount of time on both. Something might have changed inside NIntegrate when updating version 10 to 11.
UPDATE 2 (21/08): I narrowed the problem to a power. Evaluating the following code
NIntegrate[FFAuR[Power[x, 1]], {x, 0, 10}] // AbsoluteTiming
{0.990320, 0.353331}
takes 1s on the cluster and 1.1s on my on my computer. Squaring the argument of the function FFAuR somehow wrecks everything:
NIntegrate[FFAuR[Power[x, 2]], {x, 0, 10}] // AbsoluteTiming
{461.942459,0.580889}
It takes 435s to evaluate this code on the cluster while it takes only 3s on my PC! I still don't know why this happens.
UPDATE 3 (21/08): I found the solution. Mathematica 10.0 has a different symbolic preprocessing which makes the evaluation of NIntegrate take much longer. Adding Method->{"SymbolicProcessing" -> 0} at the end of NIntegrate solves the problem!
Thanks!
