26
$\begingroup$

My friend and I are trying to set up remote kernels.

He has a server at home running 1 kernel off of a Linux-based server. We're both on 8.0.

I manually set $RemoteCommand to

"ssh `3`@`1`-l `3` \"math -mathlink -linkmode Connect `4` -linkname \
`2` -subkernel -noinit >& /dev/null &\""`

and manually changed $RemoteUserName to the one he set up for me. The problem is, whenever we try and run the code, we get the following error message:

LaunchRemote::rsh: Command ssh [my username]@[his server address]-l [my username] "math -mathlink -linkmode Connect -linkprotocol TCPIP -linkname 51145@[my public IP address],51146@[my public IP address] -subkernel -noinit >& /dev/null &" may have failed (exit code 1).

The problem is, my public IP address is not the same as my private IP address, so I can't connect. How can I get Mathematica to recognize my private IP instead?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
13
$\begingroup$

Dale Roberts's solution works fine, but only when the remote machine can be seen directly from the Internet. After connection you can type

Links[]

and see

{LinkObject["[email protected],[email protected]", 3, 1], LinkObject["[email protected],[email protected]", 56, 2], LinkObject["[email protected],[email protected]", 57, 3]}

It means that the main link (the first) was opened through the ssh tunnel, but the preemptive link and the service link (the second and the third) was opened directly to the remote machine (it has 109.1.2.3 ip in this example).

If the remote machine is behind a router with port redirection this approach is unfortunately not worked. It is because MathLink naively try to open the preemptive link and the service link to the remote machine ip which is now something like 192.168.1.123. This behaviour can not be changed from extarnal options like LinkName and LinkHost.

I found great solution here: Remote Kernel Strategies. It simply redefines MathLink`CreateFrontEndLink. It was written for Mathematica 6, but it also works with Mathematica 9.

Now I can connect and see that all connections are through the ssh tunnel

Links[]

{LinkObject["[email protected],[email protected]", 3, 1], LinkObject["[email protected],[email protected]", 57, 2], LinkObject["[email protected],[email protected]", 58, 3]}

For further study, see excellent explanation in init.m in Remote Kernel Strategies.

I want to add some information about connecting to and disconnecting from the remote kernel without it's interrupting. It may be useful if you want to send a long task, turn off local machine and reconnect to the remote machine several days later.

I don't know the best place to post this information. This question is about connecting to the master kernel on the server while the subkernels are running. So I will be happy if you correct me.

I modify the last command in tunnel.sh in Remote Kernel Strategies in the following way:

/usr/bin/ssh \
 $SSH_OPTS \
 -R 127.0.0.1:$MAIN_LINK_DATA_PORT:127.0.0.1:$MAIN_LINK_DATA_PORT \
 -R 127.0.0.1:$MAIN_LINK_MESSAGE_PORT:127.0.0.1:$MAIN_LINK_MESSAGE_PORT \
 -L $PREEMPTIVE_LINK_DATA_PORT:127.0.0.1:$PREEMPTIVE_LINK_DATA_PORT \
 -L $PREEMPTIVE_LINK_MESSAGE_PORT:127.0.0.1:$PREEMPTIVE_LINK_MESSAGE_PORT \
 -L $SERVICE_LINK_DATA_PORT:127.0.0.1:$SERVICE_LINK_DATA_PORT \
 -L $SERVICE_LINK_MESSAGE_PORT:127.0.0.1:$SERVICE_LINK_MESSAGE_PORT \
 $REMOTE_KERNEL_HOST \
 "screen -wipe; \
 screen -ls | grep MathKernel -q || \
 screen -md -S MathKernel $REMOTE_KERNEL_PATH && \
 screen -md -S Respawner screen -x MathKernel; \ 
 sleep 1; \
 screen -S MathKernel -X stuff `echo -e \"\\\\\\\$ParentLink=LinkCreate[\\\\\\\"$LINK_NAME\\\\\\\",LinkMode-\\\\\\\>Connect,LinkProtocol-\\\\\\\>\\\\\\\"TCPIP\\\\\\\"]\\\\\\\;\\\r\"`; \
 sleep 20" >> $LOGFILE 2>&1

It starts screen session with MathKernel name if it isn't exist. Then it connects to the existing session by

$ParentLink=LinkCreate["$LINK_NAME",LinkMode->Connect,LinkProtocol->"TCPIP"];

Without sleep Mathematica doesn't have time to establish the connection. For disconnecting I use

SetAttributes[Disconnect, HoldAll];
Disconnect[expr___] :=
 Module[{links = {$ParentLink, MathLink`$ServiceLink,
     MathLink`$PreemptiveLink}}, $ParentLink = Null;
  LinkClose /@ links; MathLink`$PortNumber =.; SetOptions["stdout", FormatType->OutputForm];
  expr]

Then you can simply run something like this

x = Sum[EulerPhi[n], {n, 10000000}]; // Disconnect

and x=... will be evaluated after disconnection. MathLink`$PortNumber =. is necessary to connect to the same ports after disconnecting. For convenienceDisconnect can be added to init.m.

Update 1: you have to run two screen sessions and attach one to another because you can send commands only to attached sessions. You can directly connect to the screen session by screen -x MathKenel. It is useful if you want to check the state of your program from the mobile phone.

Update 2: LinkClose/@Links[] is bad because it closes links to parallel subkernels. So I explicitly specify $ParentLink, MathLink`$ServiceLink, and MathLink`$PreemptiveLink.

Update 3: Explicit port specification isn't needed.

Update 4: The terminal formatting is restored by SetOptions["stdout", FormatType->OutputForm]. Thanks this answer.

Update 5: After reconnection custom syntax highlighting is dropped. You can repair it by

Scan[ToExpression[#, InputForm, 
    Function[name, 
     SyntaxInformation@Unevaluated[name] = 
      SyntaxInformation@Unevaluated[name], HoldAll]] &, 
  Names["Global`*"]];

Update 6: Mathematica 10 has libraries which can be incompatible with our SSH. If you have an error

ssh: symbol lookup error: ssh: undefined symbol: EVP_aes_128_ctr

or something like this then reset an environmental variable $LD_LIBRARY_PATH by adding

export LD_LIBRARY_PATH=

to tunnel.sh.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for the answer. It's very helpful for me. I actually also use the Remote Kernel Strategies to connect to the remote kernel. On our university super computer, the TCPIP communication seem to have been blocked. And using the package, I was able to connect my local frontend to the remote main kernel. But the TCPIP communication between the remote master kernel and the subkernels is also blocked. Do you know how to let the communication between subkernel and main kernel also through ssh? $\endgroup$
    – xslittlegrass
    Commented Feb 3, 2015 at 1:51
  • $\begingroup$ Thank you so much for this answer. Please could you take a look at my follow up question? mathematica.stackexchange.com/questions/90051/… $\endgroup$
    – Calchas
    Commented Aug 6, 2015 at 18:07
  • $\begingroup$ No longer works in v11 either. $\endgroup$
    – M.R.
    Commented Aug 29, 2017 at 2:19
  • $\begingroup$ Does anyone have a solution that works in 13.x? $\endgroup$
    – swish
    Commented Jun 10, 2023 at 15:23
8
$\begingroup$

I've been successful connecting to my office Mac workstation using SSH port forwarding:

ssh -R 31415:127.0.0.1:31415 -R 31416:127.0.0.1:31416 [email protected] "/Applications/Mathematica.app/Contents/MacOS/MathKernel -mathlink -LinkMode Connect -LinkProtocol TCPIP -LinkName [email protected],[email protected] -LinkHost 127.0.0.1"

where [email protected] should be replaced with your SSH user name and the IP address of the remote machine. If the remote machine is running Linux, replace

/Applications/Mathematica.app/Contents/MacOS/MathKernel

with the appropriate path of the MathKernel executable.

Improve this answer
$\endgroup$
3
  • $\begingroup$ So 31415 and 31416 are ports.. ARe these default values? Can I do something similar to start a remote kernel in my notebook/script? $\endgroup$
    – dearN
    Commented Oct 7, 2012 at 13:44
  • $\begingroup$ You can change those port numbers to anything. Yes, you should be able to use the same to start a remote kernel in your notebook/script. $\endgroup$
    – partition_of_unity
    Commented Oct 7, 2012 at 23:18
  • 3
    $\begingroup$ can you give more details on how/where you use the command? $\endgroup$
    – jens_bo
    Commented Nov 6, 2012 at 0:16
2
$\begingroup$

I have never had luck with this directly, since the remote machine has to connect back to the master. However, there is a way to do it with OpenVPN -- make the two machines connect to the VPN appear to be on the same subnet.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ and just to add -- you might think that you can do port forwarding through your router. But it often doesn't work because of ISP port-blocking policies. Also, I have never gotten the ssh forwarding stuff to work, either. $\endgroup$
    – Eric Brown
    Commented Oct 7, 2012 at 5:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.