2
$\begingroup$

I need to write data in multiple files in ParallelDo loop:

 Table[s[i] = OpenAppend[StringJoin["t", ToString[i], ".dat"], 
   FormatType -> OutputForm], {i, 1, 2}]

 ParallelDo[Write[s[i], i , "\t", i + 100], {i, 1, 2}]

This leads to errors like:

Write::openx : OutputStream[t1.dat,3] is not open.

Note that writing is no trouble in a normal Do loop. What is the way to write in multiple files in ParallelDo ?

$\endgroup$
  • 2
    $\begingroup$ You are opening the file in the local kernel and trying to use the handler in the remote kernel. $\endgroup$ – rhermans Jun 22 '18 at 14:03
  • 2
    $\begingroup$ ParallelDo opens multiple, separate subkernels. Each one of these needs to have the relevant stream open in it. In the above code you're only opening the streams in the controlling kernel. $\endgroup$ – user6014 Jun 22 '18 at 14:04
  • $\begingroup$ You can use Streams[] to list all the available streams in a kernel. $\endgroup$ – rhermans Jun 22 '18 at 14:06
1
$\begingroup$

Try something like this.

LaunchKernels[];

task[n_] := Module[
  {file},
  SetDirectory[$TemporaryDirectory];
  file = OpenAppend[StringTemplate["File``.dat"][n]];
  WriteLine[file, 
   StringTemplate["Date:`1`, Kernel:`2`, Iterator:`3`"][
    DateString[], $KernelID, n]];
  Close[file]
  ]

DistributeDefinitions[task];

ParallelDo[
 task[i], {i, 8}
 ]

SystemOpen[SetDirectory[$TemporaryDirectory]]

FileNames["Fil*.dat"]
(* {"File1.dat", "File2.dat", "File3.dat", "File4.dat", \
"File5.dat", "File6.dat", "File7.dat", "File8.dat"} *)

I'm assuming all the kernels are local so have the same $TemporaryDirectory in the same filesystem.

$\endgroup$
  • $\begingroup$ Thanks for pointing out the mistake. I've done a small experiment : even just defining the streams in paralleldo loop without specifying the kernel numbers also work. How is it possible? How can mathematica figures out the right kernel for a specific set of calculation ? $\endgroup$ – Karamveer Kaur Jun 22 '18 at 15:23
  • $\begingroup$ @KaramveerKaur It's hard to tell without seen exactly what you have done. I suggest you post another question with all the necessary details to reproduce your experiment. Somebody (probably me) would be happy to help there. Let's try to keep the site well organized and keep each question about a single topic. Ok? Please notice that "moving the goalpost" is normally not well received. $\endgroup$ – rhermans Jun 22 '18 at 15:29
  • $\begingroup$ okee ... will do so ... :) $\endgroup$ – Karamveer Kaur Jun 22 '18 at 15:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.