7
$\begingroup$

I have the following code, schematically:

$runningLogFile = "data.txt";

If[FileExistsQ[$runningLogFile], Get[$runningLogFile], Export[$runningLogFile, "", "Text"];];

f[x_, y_] := Block[{},
    (*do some calculation*)
    (*put append result to $runningLogFile*)
]

LaunchKernels[$ProcessorCount];
DistributeDefinitions[...]; 
ParallelTable[f[x,y], {x, 1, 10}, {y, 0, 15}];

This code works well, except for the fact that very ocassionally (surprisingly few times in fact, but still annoying) two threads try to write the result out to $runningLogFile at the same time, and the results output get spliced together. Is there a way to do this in a thread safe manner? (perhaps similar to critical procedure in openmp)

$\endgroup$
  • $\begingroup$ I guess you could do all the writing from one kernel, ParallelEvaluate[writetolog[msg], 1] $\endgroup$ – ssch Dec 8 '12 at 14:46
  • $\begingroup$ or write to n different files and join at the end. $\endgroup$ – Ajasja Dec 8 '12 at 15:19
13
$\begingroup$

Use the function CriticalSection[{lck}, expr], for example:

CriticalSection[{logfileLock}, PutAppend[result, $runningLogFile]]

This way the result, if it's large, stays on its kernel. The other approach for synchronization is as ssch suggests, do things like this in the master kernel with shared functions, but then you need to ship the result from the subkernel to the master kernel. Another tool in the synchronization arsenal is the shared variable, which is a variable that exists on the master kernel but is visible to subkernels for reading and writing. Instead of storing a log in a file, you could store it in a shared variable, but this also has the drawback that the value must be sent between master and subkernel.

CriticalSection and shared variables are documented in the Parallel Tools Virtual Shared Memory tutorial. SharedFunction is documented in the SharedFunction reference page.

$\endgroup$
  • $\begingroup$ logfileLock must be in a list, i.e. CriticalSection[{logfileLock}, ...]. $\endgroup$ – WReach Dec 8 '12 at 16:20
  • $\begingroup$ @WReach: Thanks, I corrected the usage. $\endgroup$ – Joel Klein Dec 8 '12 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.