# Error when implementing StartProcess[] in loops (Do, While, Table)

I'm using Mathematica to control another program (Surface Evolver) through the terminal (ie using StartProcess[]; and WriteLine[];). The programs communicate by writing and reading text files (Export[]; and Import[]; respectively). The script I wrote is packaged in Block[]; (Module[]; yields the same results) and run iteratively in a While[] loop (I've also used Do[] and Table[], same results). Everything works great when I run small batches of iterations. I don't think I'm running out of memory (I clear the sys cache and wipe variables within the module every iteration to prevent memory leaks because I've seen some chatter about that in other threads). However, at about n=100 I'll get an error

OpenWrite::noopen: Cannot open in:100


EDIT

Okay, this should be more manageable, I've isolated the source of the error to the repeated creation of processes. It seems like the error is not MTHM per se, but an inability to open a new system process.

Dynamic[n]
Do[Module[{TestProcess},
TestProcess = StartProcess[$SystemShell]; WriteLine[TestProcess, "echo example line"]; KillProcess[TestProcess]; Pause[0.1]; Clear[TestProcess]; ClearSystemCache[]; ];,{n, 1, 1000}];  END EDIT System Info: 1. Mac OS X Mavericks 2. Mathematica 10 3. Screen Saver is off • You second code-block is not complete. The start seems to be missing and the ,{n,1,1000}]; belongs to nothing. Otherwise, nice that you could break the problem down. It is more likely that someone can help you with this. Oct 10 '14 at 0:08 • Fixed, thanks - that is the hope! Oct 10 '14 at 1:41 • Mathematica 10 under Linux: bunch of errors after ~900 cycles. General::stream: -- Message text not found -- ($Failed) and so on. Oct 10 '14 at 1:44

The problem is that the stdin stream for each process is being left open and needs to be closed.

To demonstrate the issue, we create a single process:

$process = StartProcess[$SystemShell];
WriteLine[$process, "echo example line"]; KillProcess[$process];


The call to WriteLine implicitly creates a stream object. Even though the process has been killed, the stdin stream object remains (named in:0 in this case):

Streams[]


We need to manually close the stream (see the sidebar below for why we have to use "StandardOutput" to get the stdin stream):

Close[ProcessConnection[\$process, "StandardOutput"]];

Streams[]


Also, had we obtained any explicit references to any of the other standard process streams, we would need to close them manually as well.

This would be a good thing for the documentation to mention in the Possible Issues section :)

Sidebar (EDIT: the problem described below has been fixed in version 10.0.2)

I will mention in passing that ProcessConnection has the stream names for stdin and stdout reversed. We must reference stdin using the stream name "StandardOutput" and stdout using "StandardInput". This confusion is evident in examples given in the documentation:

Note how the example is writing the string "echo line 1" to the shell's stdin, but the stream name used was "StandardOutput". Beware.

• Note: I tested this on Windows 7, which exhibited the problem as well albeit at a much higher iteration count. I expect the solution to be the same on MacOSX, but I am not in a position to test it directly myself at the moment. Oct 11 '14 at 13:10
• The problem with swapped stream names has been fixed in version 10.0.2. Dec 10 '14 at 22:33

We had a bug like this on v. 10: it specifically stopped running new processes after you created 100 of them. This was fixed in version 10.0.1. I recommend testing it there, as version 10.0.1 has a number of improvements with respect to StartProcess.

I can't test your example in MacOS right now, so please let me know if this isn't working on v. 10.0.1.

• +1. With respect to the stream names... I would agree if the stream names were simply "Input" and "Output". But I would argue that the qualifier Standard in "StandardInput" implies that it is being seen from the perspective of the process. After all, Mathematica's own stdin and stdout streams play no role here (i.e. brand new streams are created for the connection). However, what is done is done. It would be harmful to change it now. Perhaps it would make a fine addition to the Possible Issues section of the documentation :) Oct 13 '14 at 17:47