2
$\begingroup$

I've got a long calculus to do with Mathematica. I've Parallelized the code using local kernels and remote kernels. I'm using a ParallelDo loop with an Export function instead a ParallelTable to save the intermediate results. However I've got some issues while Exporting the intermediate results computed by the remote kernels to the local machine. My goal is to save all the intermediate results in the same directory on the local machine

My code looks like that:

dir = NotebookDirectory[];
ParallelDo[Module[{temp},
           temp = DensityPlot[Myfunction[x,y,t],{x,-1,1},{y,-1,1}];
           Export[dir <> "localsubdir/" <> ToString[NumberForm[t, {5, 5}]] <> ".png", temp]], {t, 0, 0.02,0.0001}]

For the local kernels it's working, but as soon as a remote kernels is involved. I've got an error of non-existing folder:

Directory /mylocalexpordirectory/ does not exist. >>

This is logical since the directory "mylocalexportdirectory" doesn't exists on the remote machine. So I looked for some solutions to export intermediate results through ssh:

Export["!ssh user@localmachine cat /mylocalexpordirectory/temp.png", temp]

But I've got the following output with no exportation made:

OutputStream["!ssh user@localmachine cat /mylocalexpordirectory/temp.png", 364]

Any ideas?

$\endgroup$
  • $\begingroup$ Is the Export operation itself a significant portion of your evaluation time or could all Export operations be done from a single kernel? $\endgroup$ – Mr.Wizard Jun 24 '15 at 7:13