I'm trying to create a custom progress indicator for a parallelized computation, using Dynamic
and SetSharedVariables
to display and update the progress indicator during the calculation. Two questions:
What is the best way to allow parallel kernels to update the progress? Should I have a function which accepts a
Dynamic[x_]
argument and do something like this?Monitor[PerformCalculations[Dynamic[progress]], ProgressIndicator[Dynamic[progress]]]; (* or this: *) ProgressIndicator[Dynamic[progress]] PerformCalculations[Dynamic[progress]] (* note: 'progress' here will be a list or some other custom structure for indicating various aspects of the progress of each kernel *) PerformCalculations[Dynamic[progress_]] := Module[{calc, ...}, SetSharedVariable[progress]; calc[stuff_] := Module[{...}, ...; (* how does this part work? *) UpdateProgress[Dynamic[progress]]; ]; ParallelMap[calc, data]; ];
How do I avoid race conditions of shared variables? A quick test indicates that shared variables have some caveats when running with parallel kernels:
I would have expected the
test = Append[test, ...]
expression to be evaluated atomically so that each kernel would build on the results of the kernels that had already executed. However it seems like the kernels read the value oftest
before writing it, so there is a race condition that causes the final value to be simply{8}
. What is the best way to avoid this issue?