Wolfram Light Weight Grid and parallel computing

Question

I have attempted to set up and use the Wolfram Lightweight Grid Manager so I can do parallel computing from Mathematica using 2 remote processors on another machine on my local network.

My computing environment includes:

Computers:

Name: iMac 6
27" iMac, 3.06 GHz Intel Core 2 Duo; 4 GB 1067 MHz DDR3.

Mathematica version: 8.0.1.0

Name: iMac 7 27" iMac, 3.06 GHz Intel Core 2 Duo; 4 GB 1067 MHz DDR3.

Network: Apple Time Capsule -- note the 2 iMacs connect to the Time Capsule by hard wired ethernet connections. No internal firewall prevent access to "iMac 7".

As my license supports 4 processors for parallel computing, it should enable me to identify the 2 remote processors and then distribute computations across them and the 2 on my main machine.

I've installed Wolfram LightWeight Grid Manager on iMac 6, my main Mathematica machine.

When I look at Preferences>>Parallel it looks like the following:

You can see that it identifies the kernels at IP address 10.0.10.22, my iMac 7 machine, but in the "Parallel Kernel Status" window only the kernels on iMac 6 appear as available.

I run the following in a notebook to attempt to identify and launch the remote kernels:

SetSystemOptions["ParallelOptions" -> "MathLinkTimeout" -> 120]; (* SETTING AS ADVISED BY PREMIER SUPPORT *)

Needs["LightweightGridClient`"]

$ProcessorCount
ParallelEvaluate[$ProcessID]
ParallelEvaluate[$MachineName]
$MaxLicenseSubprocesses

2
{50948, 50950}
{"iMac-6", "iMac-6"}
4

RemoteServicesAgents[]
{"http://10.0.10.22:3737/WolframLightweightGrid/Manager"}

(* THE ABOVE SEEMS TO MAKE SENSE BECAUSE THE IP ADDRESS IF THE REMOTE MACHINE IS: 10.0.10.22 *)

$KernelCount

2

CloseKernels[] (* JUST TO GET A CLEAN START *)


LaunchKernels[] (* GIVES THE FOLLOWING ERROR *)

RemoteKernelOpen::lwgconnect: Unable to connect to http://10.0.10.22:3737/WolframLightweightGrid/Manager.  Check network connectivity and the spelling of the hostname or URL of the remote computer. Confirm that a Lightweight Grid Manager is running on the remote computer. >>

{KernelObject[17, "local"], KernelObject[18, "local"], $Failed, $Failed}

Apparently, Mathematica can see the 2 processors on my local machine (iMac 6 IP address: 10.0.10.3), but cannot reach the additional processors on the remote machine (iMac 7:IP address: 10.0.10.22).

The Wolfram Lightweight Grid Manager opens up in a browser. It has a "Licensing" tab that asks for a login, but interestingly does not accept either the administrative login for the computer or any of the 2 user names and passwords that one has to set up when one installs and configures the Lightweight Grid Manager. The how to video (see link) doesn't make this clear.

Not certain what to try next.

Do I need to install some kind of grid client software on the remote computer? Do I need to login to the remote computer from the local one? Do I need to have Mathematica installed on the remote computer?

I've gone through the documentation, and I have to say I have not found it the clearest that Wolfram has produced.

I seem to have missed something. Does anyone have an idea of how to configure this so I can launch remote kernels and access them?

Answer 1

I've discovered (at least some of) the answer to this question.

1. One needs to install the Wolfram lightweight grid manager on local AND remote computers. This seems a bit confusing as the title of the program that one sees during the installation, refers to this application as "Wolfram gridMathematica Server with Wolfram Lightweight Grid Manager". A bit clearer nomenclature i.e. "...server/client manager" would, in my mind make this clearer.

2. The login on the "Licensing" tab of the Wolfram Lightweight Grid Manager" that opens in a browser requires the user name "admin". This also seems a bit confusing, because while they refer to this user name in the set up, the user does not enter it. If only "admin" works as a user name in this instance why require the user to enter it at all? I thought it wanted either the administrator's name for the computer or the login for the grid manager. Also, I don't understand why it requires two sets of user names and passwords to setup and run the manager.

3. Apparently, while a desktop license of Mathematica includes 2 "Controlling" processes and 4 "Computing" process (see the following from the Wolfram User Portal:

The number of Processes allowed is formated as (X/Y), where X is the number of Controlling Processes and Y is the number of Computation Processes.

A Controlling Process indicates the number of simultaneous network users of "seats" allowed. A Controlling Process handles input, output, and scheduling for the Computation Processes.

A Computation Process indicates the number of computations that may be run simultaneously.)

in discussing this with support earlier today, it appears Mathematica licensing requires Computation Processes to reside on the same machine as the Controlling Process.

It seems one can have two copies of the same license installed on different machines, but one can only use one of the copies at a time and one can only access the Computation Processes for the license if they reside on the same machine as a copy of the license.

This seems a bit antiquated given Sun Micro Systems' (via John Gage's) recognition a couple of decades ago that "The network is the computer." Why should the location of the Computation Processes matter?

So if I have the correct information from support, to get access kernels on a remote machine one needs a full copy of Mathematica installed on the remote machine.

Update: I just installed a full desktop version of Mathematica on my remote machine and can now see and access its 2 processors from the Preferences >> Lightweight Grid window and see them in action on the Parallel Kernel Status window:

To my mind, this licensing approach does not align well with the vision and promise of distributed processing, which sought to enable people to access computing resources anywhere on a network. While advanced desktop machines have started to have 4 processors, I'd wager most machines still have only one or two. Meanwhile lots of people may have a couple of desktops or some combination of desktops and laptops available. Just a shame we can't make use of them.

Maybe it makes sense to study up on CUDA and GPU.

Hope this helps.

Jagra

---

Some additional information from Mathematica support relative to my initial thoughts:

"One needs to install the Wolfram lightweight grid manager on local AND remote computers".

This is incorrect, the lightweight grid manager is only needed on the remote computers (or the computers that will be supplying the kernels for parallel computation). The only time the lightweight grid manager will be required on the local machine is if other machines will be launching kernel from this local machine.

"...it appears Mathematica licensing requires Computation Processes to reside on the same machine as the Controlling Process".

This is also incorrect. The controlling process is only required on the job submission node. The computation nodes can reside across multiple machines which do not need to have another controlling process. (Basically, they have kernels and no front ends).

"It seems one can have two copies of the same license installed on different machines."

This is incorrect as well. Unless you are running a license manager or the lab version (which will not run gridMathematica), you cannot run the same license on two machines at any one time. The passwords are machine dependent. You will need to call us anytime you need to switch machines.