Why is the execution time getting smaller and smaller every time?

Question

Why is the execution time to compute the value of sum getting smaller and smaller every time in the following code? And how to get consistency? Thank you very much for your help.

In[18]:= Quit[]

In[1]:= atime = AbsoluteTiming;

In[2]:= m = 1350; k = 10^4;

In[3]:= LaunchKernels[] // atime

Out[3]= {3.11965, {"KernelObject"[1, "local"], 
  "KernelObject"[2, "local"], "KernelObject"[3, "local"], 
  "KernelObject"[4, "local"], "KernelObject"[5, "local"], 
  "KernelObject"[6, "local"]}}

In[4]:= ClearSystemCache[]
sum = ParallelSum[
    1/be Log[10^8 - (be - 1)^2/4] // N[#, k] &, {be, 2, m}]; // atime

Out[5]= {0.905762, Null}

In[6]:= ClearSystemCache[]
sum = ParallelSum[
    1/be Log[10^8 - (be - 1)^2/4] // N[#, k] &, {be, 2, m}]; // atime

Out[7]= {0.359625, Null}

In[8]:= ClearSystemCache[]
sum = ParallelSum[
    1/be Log[10^8 - (be - 1)^2/4] // N[#, k] &, {be, 2, m}]; // atime

Out[9]= {0.250268, Null}

In[10]:= ClearSystemCache[]
sum = ParallelSum[
    1/be Log[10^8 - (be - 1)^2/4] // N[#, k] &, {be, 2, m}]; // atime

Out[11]= {0.167734, Null}

In[12]:= ClearSystemCache[]
sum = ParallelSum[
    1/be Log[10^8 - (be - 1)^2/4] // N[#, k] &, {be, 2, m}]; // atime

Out[13]= {0.0718937, Null}

In[14]:= ClearSystemCache[]
sum = ParallelSum[
    1/be Log[10^8 - (be - 1)^2/4] // N[#, k] &, {be, 2, m}]; // atime

Out[15]= {0.0620984, Null}

Answer 1

It looks like ClearSystemCache[] will only clear the cache of the master kernel, but not that of any of the parallel kernels. This could have been implemented differently but maybe there are good reasons to not automatically do this. The behavior is not too surprising if you bring to mind that the parallel kernels are more or less independent instances of the kernel which only do interact weakly with the provided Parallel* functions. Using:

ParallelEvaluate[ClearSystemCache[]]

will clear the caches of the parallel kernels and show the desired result (repeatable "slow" evaluation) for your example. For some cases, it might be necessary to clear the cache on both the parallel kernels and the master kernel to achieve repeatable timings.

Answer 2

I am posting the response that I have just received from Wolfram Technical Support, which appears to confirm Albert's suggestion:

Hello Prof. Pinelis,

There are two issues:

1) the subkernels also cache results 2) ParallelSum distributes the tasks across the subkernels in a "random" manner

The combination of these two issues is that the absolute timing appears to decrease each time you evaluate the expression, down to some lower limit. This lower limit is achieved when all subkernels have cached all intermediary results. They won't cache all intermediary results until ParallelSum has been evaluated enough times, such that the "random" distribution of tasks has covered the entire set of subkernels, i.e. each subkernel has seen each possible iterator and cached the results.

To get around the caching in the subkernels, you can evaluate the command ParallelEvaluate[ClearSystemCache[]]. This clears the cache in the subkernels memory.

Sincerely,

Kevin Daily

Wolfram Technical Support Wolfram Research, Inc. http://support.wolfram.com/