8
$\begingroup$

I'm trying to create a custom progress indicator for a parallelized computation, using Dynamic and SetSharedVariables to display and update the progress indicator during the calculation. Two questions:

  1. What is the best way to allow parallel kernels to update the progress? Should I have a function which accepts a Dynamic[x_] argument and do something like this?

    Monitor[PerformCalculations[Dynamic[progress]],
      ProgressIndicator[Dynamic[progress]]];
    (* or this: *)
    ProgressIndicator[Dynamic[progress]]
    PerformCalculations[Dynamic[progress]]
    
    (* note: 'progress' here will be a list or some other custom structure 
       for indicating various aspects of the progress of each kernel *)
    
    PerformCalculations[Dynamic[progress_]] := Module[{calc, ...},
        SetSharedVariable[progress];
        calc[stuff_] := Module[{...},
          ...;
          (* how does this part work? *) UpdateProgress[Dynamic[progress]];
         ];
        ParallelMap[calc, data];
      ];
  2. How do I avoid race conditions of shared variables? A quick test indicates that shared variables have some caveats when running with parallel kernels:

    I would have expected the test = Append[test, ...] expression to be evaluated atomically so that each kernel would build on the results of the kernels that had already executed. However it seems like the kernels read the value of test before writing it, so there is a race condition that causes the final value to be simply {8}. What is the best way to avoid this issue?

$\endgroup$

1 Answer 1

9
$\begingroup$

I have my own progress indicator function in my toolbag. That has this form:

progressBar[dyn_, total_] :=
  Print@Row[
    { ProgressIndicator[dyn, {0, total}], " "
    , Dynamic@NumberForm[100. dyn/total, {\[Infinity], 2}]
    , "% ", dyn
    }
  ]

I use it in this way:

testFunction[data_]:=Module[{i=1,len=Length[data]},
    SetSharedVariable[i];
    progressBar[Dynamic@i,len];
    ParallelMap[(i++;Pause[0.1];#)&,data]
]

Now let's call the testFunction over the list Range[100]

testFunction[Range[100]]

progress indicator

So with these two steeps (SetSharedVariable[i];progressBar[Dynamic@i,len];) and with the progress indicator function (progressBar), it's very easy to implement it. It's the way I do.

Tip

Sometimes to work with shared variables, it's important to use SetSharedFunction, and this post explain how to use it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.