How do I run scripts on networked computers directly from Mathematica?

Question

The answer to an earlier question,

Computers running Wolfram Lightweight Grid Manager won't fall asleep,

left me with a new problem. To review, I have a computing environment comprised of:

• 27" iMac with dual cores running Mac OS X 10.6.8 and
• 2 XServes each with 2 quad cores running Max OS X Server 10.6.8
• All networking by ethernet to a Time capsule.

As this question has more general application than the earlier question, I thought it merited its own post.

...

I need to launch and shut down a service (in my case Wolfram Lightweight Grid Manager but this really applies to any service, program, or application) which needs to run on a server (or more generally, any other machine) attached to my network (or again more generally, any network to which one has authorized access).

So, ideally I'd like to do the following:

• Access and log on to (a) remote computer(s) on a local network directly from Mathematica;

• Launch a script on the remote computer that will in turn launch Wolfram Lightweight Grid Manager;

• Launch available kernels;

• Run parallel processing code;

• Close remote kernels; and

• Launch a script on the remote computer to close Wolfram Lightweight Grid Manager.

---

Can I do all of this from within Mathematica?

How?

---

Run[] enables one to run a command line program:

But how do I get to a remote machine, especially as it does not likely have a fixed IP address? ;-(

I would appear to need a way to identify machines on the network (I do have machine names and log on credentials).

I've very rarely done this sort of thing.

---

Could a script launched locally by the Run[] function access the remote machines and launch a script on them?

---

That would make it easy.

References or examples welcome.

My Unix script never amounted to much, so any guidance welcome.

It also, occurs to me that one could do this via JLink, but as the little I knew of Java has generally drifted out of my memory a more direct solution would do me better.

Answer 1

Once setup you should be able to do all of what you want from Mathematica without resorting to shell scripts.

This isn't Apple specific, but hopefully this will get you most of the way and uses the Remote Kernel method rather than the lightweight grid manager:

1. Create an ssh key if you haven't already, using ssh-keygen

2. Copy that key to all the remote hosts you want to use using ssh-copy-id hostname Check you can login without supplying a password to each host with ssh hostname

3. Define the remote machines to be used and the number of kernels using menu item: Evaluation->ParallelKernelConfiguration: [TAB] Parallel [TAB] Remote Kernels

which requires a hostname and possibly a custom launch command. I needed to specify the fullpath the Mathematica kernel, which in my case was /usr/local/bin/math giving:

    ssh -x -f -l `3` `1` /usr/local/bin/math -mathlink -linkmode Connect `4` -linkname '`2`' -subkernel -noinit

1. The kernels you have defined using this method, as well as your local kernels, are then held in $ConfiguredKernels. They can all be launched using LaunchKernels[]

2. The control of code execution can then be done with the usual Mathematica parallel evaluation functions and the kernels can be closed jointly or separately using CloseKernels[]. The current running kernels can be listed by Kernels[].