Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

Possible Duplicate:
Computing Many Slow I/O Operations

I'm launching an external command with the Mathematica front end that takes two arguments:

Run["program","arg1","\"S:\\filelocation.data\""]

This operation take ~10 minutes to complete. While it's running, all other calculations will not evaluate until the external program completes.

In theory, I would want the front end to still be able to evaluate and even to run the kill command for the program

Run["program,"kill"]

I have an 8-core machine, so a Parallel solution would work.

Thanks.

UPDATE:

So, as answered below, ParallelSubmit works well if submitted together:

enter image description here

But still doesn't free up the notebook to evaluate other cells:

enter image description here

share|improve this question

marked as duplicate by rm -rf Jan 29 '13 at 21:16

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

    
I can't play with this right now but I think you will want to look at ParallelSubmit. –  Mr.Wizard Oct 23 '12 at 13:34
1  
Isn't this a duplicate of mathematica.stackexchange.com/q/2874/5 ? –  rm -rf Oct 23 '12 at 19:23
    
@rm-rf It seems so. However, Mr.Wizards answer contains another (if not programatic) approach I didn't know about. –  sebhofer Oct 24 '12 at 8:21
    
@rm-rf do you think this is one that is a legit Merge? The answers here appear to be neutral. –  Mr.Wizard Oct 25 '12 at 0:27
    
@Mr.Wizard I don't see a reason to merge. Simply closing should be sufficient. Merging two questions each with accepted answers will result in one answer losing the accept (in this case, sebhofer's). –  rm -rf Oct 25 '12 at 1:28

2 Answers 2

up vote 3 down vote accepted

As Mr.Wizard pointed out, you can do this with ParallelSubmit. But you need a little more, and that's the (not so) tricky part as it is not very well documented. I think something like the following should work for you:

Needs["Parallel`Developer`"]
f[x_] := (Pause[x]; x)
LaunchKernels[1]
DistributeDefinitions[f]
eid = ParallelSubmit[f@5]
QueueRun[]

Now you can go do some other work. When you think the external program has finished you do

WaitNext@{eid}

to collect the result.

Let me briefly explain what happens in the above code: First, I launch as many slave kernels as external functions I want to run in parallel (in this case only 1). Then the definition of f is distributed to all slave kernels. The code will also work without this step, you will not gain any speedup, however. (If the definition of f is not distributed, the slave kernels just return f@5 to the master, who will evaluate it and therefore pause for 5 seconds.) ParallelSubmit submits the computation to the queue. At this point the slave kernels are still idling. The crucial step is to run QueueRun, which is loaded from the Developer context. It starts the parallel queue but returns immediately, without waiting for results from the slave kernels. Results are collected by WaitNext (or WaitAll if you want), but as the name says, this does not return until it gets the next result from the slaves, so you can't run other computations at the same time.

To address your problem a bit more precisely, the following worked for me (on Linux that is):

Needs["Parallel`Developer`"]
LaunchKernels[2]
cmd := "echo " <> ToString[$KernelID] <> " >> foo.txt"
DistributeDefinitions[cmd]
eid = {ParallelSubmit@Run[cmd], ParallelSubmit@Run[cmd]}
QueueRun[]
WaitAll[eid]

As I understand it, the functionality in the Parallel`Developer context evolved from the "Parallel Computing Toolkit" by Roman Maeder, which was at some point integrated into main MMA. Unfortunately the documentation largely got lost in the process (which is really a shame!), but it is still available at Wolfram. The example section is definitely worth a read! Note that some commands where renamed though.

share|improve this answer
    
So this works lovely if I ParallelSubmit everything, but if I launch this process and then go down the notebook and try to evaluate 2+2 the evaluation won't return until the external command stops. –  kale Oct 23 '12 at 18:18
    
Did you actually read the answer above the last example?? (I admit I should have put more thought into the last example though... editing) –  sebhofer Oct 23 '12 at 18:37
    
This looks interesting but how do you know when the background task has completed? –  Mr.Wizard Oct 23 '12 at 18:47
    
@Mr.Wizard Good question, I can't remember :) I'll have to think about it. Also, there may be a hint in the examples (see edit above). –  sebhofer Oct 23 '12 at 18:51
    
@Mr.Wizard I think there are several ways to check when the calculations are finished. One of them is to check $QueueLength. Also there is DoneQ, but I couldn't figure out how that works right now. –  sebhofer Oct 23 '12 at 19:09

Parallel Kernels for separate Notebooks

Perhaps you simply want to run two kernels in parallel. You can do this by:

  1. Open Evaluation > Kernel Configuration Options... and set up more than one kernel.

  2. Assign a different kernel to each of two Notebooks using Evaluation > Notebook's Kernel

From there you can run your slow code in one Notebook and do your interactive programming in the other.

Mathematica graphics


"Dialogs" on a single Kernel

I guess the main point of my question is, if I'm launching an external command, even if I am waiting for feedback from it, it shouldn't take up a kernel's resources ... also not to mention that I'm being a complete jerk launching two kernels when we only have 2 on our network seat.

Perhaps your best option in that case is to run calculations in a kernel dialog while the background task is momentarily paused. On my system the shortcut key for Evaluation > Evaluate in Subsession is F7.

As an example try:

Do[Pause[1], {i, 50}];

And while it is running put the cursor in a separate cell containing:

Print[i]

And press F7.

Every time you select that cell and press F7 you should get the current value of i from the evaluating Do loop.

Mathematica graphics

share|improve this answer
    
I think ParallelSubmit works quite well as shown in my answer. –  sebhofer Oct 23 '12 at 18:41
    
@sebhofer my appologies, I didn't read your answer well and missed QueueRun. I shall try it now; hopefully it works on v7. –  Mr.Wizard Oct 23 '12 at 18:46
    
...and I thought you strive for conciseness :) –  sebhofer Oct 23 '12 at 18:46
    
I think it will –  sebhofer Oct 23 '12 at 18:46
2  
@kale You will definitely have to launch another kernel. But you will still only use 1 "Controlling" process and 1 "Computing" process. For my license there is a 1:2 ration between those two, so you might have 4 computing processes available for your site. –  sebhofer Oct 23 '12 at 19:26

Not the answer you're looking for? Browse other questions tagged or ask your own question.