1
$\begingroup$

Question: Why does the performance of ParallelTable within a Module drop so horribly when the module is RERUN several times? Actually, I am not interested in parallel computing here, but on what is being blown up and causes the parallel computations to take longer when you rerun the module. I also used Block and the slow-downs disappear, but I dont understand why. Is it because of the generation of unique local variables in Module and its scoping? Do these variables get stored somewhere, need to be checked every run for creating unique new ones and cause slower computation somehow when combined with automatic parallel programs? If yes, then it is kind of weird, since modules without automatic programs seem to be ok, see comparison below.

Context: Just for fun, I wanted to test ParallelTable (I have 4 kernels) within a module and check its time performance. Consider the following modules

LaunchKernels[]
Compute[data_, 1] := Module[
   {n1, n2},
   {n1, n2} = Dimensions[data];
   Table[Total[data[[i11]]*data[[i12]], Infinity], 
     {i11, n1}, {i12, n1}, {d, 1, n2}]
   ];
Compute[data_, 2] := Module[
   {n1, n2},
   {n1, n2} = Dimensions[data];
   ParallelTable[
    Total[data[[i11]]*data[[i12]], Infinity], 
      {i11, n1}, {i12, n1}, {d, 1, n2}]
   ];

Its not really important, what is computed, but the time performance discussed later after rerunning. As an example, just create data as a 2 by 10 matrix and check identical computations of the normal module Compute[data,1] and the parallelized Compute[data,2]

n1 = 2;
n2 = 10^1;
data = RandomInteger[{0, 10}, {n1, n2}];
Compute[data, 1] == Compute[data, 2]

(*True*)

First run: Now, just do some calculation time check for increasing the number of rows and columns and plot the results (y axis is computation time, x axis are number of columns, blue: one row, orange: two rows)

n1max = 2;
n2max = 10^1;
t = ConstantArray[, {n1max, n2max}];
Do[
 tloc = {};
 dataloc = RandomInteger[{0, 10}, {n1, n2}];
 Do[
  AppendTo[tloc, AbsoluteTiming[Compute[dataloc, i];][[1]]]
  , {i, 2}
  ];
 t[[n1, n2]] = tloc;
 , {n1, n1max}
 , {n2, n2max}
 ]
plot = 
  Table[
    ListPlot[t[[;; , ;; , i]], 
      PlotRange -> {0, Max[t]}, 
      PlotLabel -> "Compute " <> ToString[i], 
      Joined -> True, 
      PlotLegends -> Automatic], 
    {i, 2}] // TableForm

first run

Rerun: now, rerun the test just several times (here just 10 times) and look at the last evaluated time performance of the module with ParallelTable, it is now an order of magnitude slower while the the module with the standard Table shows naturally no changes!

nrerun = 10;
Do[
  t = ConstantArray[, {n1max, n2max}];
  Do[
    tloc = {};
    dataloc = RandomInteger[{0, 10}, {n1, n2}];
    Do[
      AppendTo[tloc, AbsoluteTiming[Compute[dataloc, i];][[1]]]
      , {i, 2}
      ];
    t[[n1, n2]] = tloc;
    , {n1, n1max}
    , {n2, n2max}
    ]
  , {rerun, nrerun}
  ]
plot = 
  Table[
    ListPlot[t[[;; , ;; , i]], 
      PlotRange -> {0, Max[t]}, 
      PlotLabel -> "Compute " <> ToString[i], 
      Joined -> True, 
      PlotLegends -> Automatic], 
    {i, 2}] // TableForm

rerun

If you keep rerunning or just doing more computations with the modules, the time performance keeps getting worse.

Question: Why does the performance of ParallelTable within a Module drop so horribly when the module is RERUN several times? Actually, I am not interested in parallel computing here, but on what is being blown up and causes the parallel computations to take longer when you rerun the module. I also used Block and the slow-downs disappear, but I dont understand why. Is it because of the generation of unique local variables in Module and its scoping? Do these variables get stored somewhere, need to be checked every run for creating unique new ones and cause slower computation somehow when combined with automatic parallel programs? If yes, then it is kind of weird, since modules without automatic programs seem to be ok, as in the example above.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.