# Evaluate while external command is being run [duplicate]

Possible Duplicate:
Computing Many Slow I/O Operations

I'm launching an external command with the Mathematica front end that takes two arguments:

Run["program","arg1","\"S:\\filelocation.data\""]


This operation take ~10 minutes to complete. While it's running, all other calculations will not evaluate until the external program completes.

In theory, I would want the front end to still be able to evaluate and even to run the kill command for the program

Run["program,"kill"]


I have an 8-core machine, so a Parallel solution would work.

Thanks.

UPDATE:

So, as answered below, ParallelSubmit works well if submitted together:

But still doesn't free up the notebook to evaluate other cells:

-

## marked as duplicate by rm -rf♦Jan 29 '13 at 21:16

I can't play with this right now but I think you will want to look at ParallelSubmit. –  Mr.Wizard Oct 23 '12 at 13:34
Isn't this a duplicate of mathematica.stackexchange.com/q/2874/5 ? –  rm -rf Oct 23 '12 at 19:23
@rm-rf It seems so. However, Mr.Wizards answer contains another (if not programatic) approach I didn't know about. –  sebhofer Oct 24 '12 at 8:21
@rm-rf do you think this is one that is a legit Merge? The answers here appear to be neutral. –  Mr.Wizard Oct 25 '12 at 0:27
@Mr.Wizard I don't see a reason to merge. Simply closing should be sufficient. Merging two questions each with accepted answers will result in one answer losing the accept (in this case, sebhofer's). –  rm -rf Oct 25 '12 at 1:28

As Mr.Wizard pointed out, you can do this with ParallelSubmit. But you need a little more, and that's the (not so) tricky part as it is not very well documented. I think something like the following should work for you:

Needs["ParallelDeveloper"]
f[x_] := (Pause[x]; x)
LaunchKernels[1]
DistributeDefinitions[f]
eid = ParallelSubmit[f@5]
QueueRun[]


Now you can go do some other work. When you think the external program has finished you do

WaitNext@{eid}


to collect the result.

Let me briefly explain what happens in the above code: First, I launch as many slave kernels as external functions I want to run in parallel (in this case only 1). Then the definition of f is distributed to all slave kernels. The code will also work without this step, you will not gain any speedup, however. (If the definition of f is not distributed, the slave kernels just return f@5 to the master, who will evaluate it and therefore pause for 5 seconds.) ParallelSubmit submits the computation to the queue. At this point the slave kernels are still idling. The crucial step is to run QueueRun, which is loaded from the Developer context. It starts the parallel queue but returns immediately, without waiting for results from the slave kernels. Results are collected by WaitNext (or WaitAll if you want), but as the name says, this does not return until it gets the next result from the slaves, so you can't run other computations at the same time.

To address your problem a bit more precisely, the following worked for me (on Linux that is):

Needs["ParallelDeveloper"]
LaunchKernels[2]
cmd := "echo " <> ToString[$KernelID] <> " >> foo.txt" DistributeDefinitions[cmd] eid = {ParallelSubmit@Run[cmd], ParallelSubmit@Run[cmd]} QueueRun[] WaitAll[eid]  As I understand it, the functionality in the ParallelDeveloper context evolved from the "Parallel Computing Toolkit" by Roman Maeder, which was at some point integrated into main MMA. Unfortunately the documentation largely got lost in the process (which is really a shame!), but it is still available at Wolfram. The example section is definitely worth a read! Note that some commands where renamed though. - So this works lovely if I ParallelSubmit everything, but if I launch this process and then go down the notebook and try to evaluate 2+2 the evaluation won't return until the external command stops. – kale Oct 23 '12 at 18:18 Did you actually read the answer above the last example?? (I admit I should have put more thought into the last example though... editing) – sebhofer Oct 23 '12 at 18:37 This looks interesting but how do you know when the background task has completed? – Mr.Wizard Oct 23 '12 at 18:47 @Mr.Wizard Good question, I can't remember :) I'll have to think about it. Also, there may be a hint in the examples (see edit above). – sebhofer Oct 23 '12 at 18:51 @Mr.Wizard I think there are several ways to check when the calculations are finished. One of them is to check $QueueLength. Also there is DoneQ`, but I couldn't figure out how that works right now. –  sebhofer Oct 23 '12 at 19:09