7
$\begingroup$

I have a remote kernel on a server that I would like to use to process some data. The connection works, i.e, I'm able to use the local front end and the remote kernel. However, I cannot figure out how should I access from the remote kernel the data that are on my local machine.

I thought I could simply use Import with the path to my computer and to the file that I want to import, but I didn't succeed. Something like:

Import["computerName@ip_address:/Users/myUserName/Desktop/data.txt","Table"]

Import::nffil: File not found during Import. >>

$\endgroup$
3
  • $\begingroup$ Import the data in the frontend and do DistributeDefinitions? $\endgroup$
    – Ajasja
    Jan 18, 2013 at 16:33
  • $\begingroup$ @Ajasja I looked at the documentation for DistributeDefinitions, but I don't get it how I should use it. For instance, f[n_] := PrimeQ[2^n - 1] and then DistributeDefinitions[f]; should transport the definition to all kernels. But when I try f[2] from the FE connected to the remote kernel the output is f[2] instead of True. $\endgroup$
    – VLC
    Jan 18, 2013 at 17:13
  • $\begingroup$ Ahh, sorry I thought they were slave kernels (for parallel computations), but you have the FE connected directly to a remote kernel. Then you have to share a disk or a drive or copy your data on some webserver... $\endgroup$
    – Ajasja
    Jan 18, 2013 at 17:17

1 Answer 1

7
$\begingroup$

After some fiddling I found a solution to directly access my local files from the remote kernel via SSH. If localuser is the user name on the machine that runs the front end on which are also the data you want to import from the remote kernel and 111.111.111.11 is the IP address of the same machine, you just need to type:

Import["!ssh [email protected] cat /Users/localuser/Desktop/data.txt", "Table"]

To make this work smoothly you first need to set up public key authentication over SSH. Let remoteuser be the user name on the machine that runs the remote kernel and let 222.222.222.22 be the IP address of the remote machine.

On your local machine launch the terminal:

  1. Generate a public key:
    • ssh-keygen -t rsa
  2. Ok, when asks to save it in:
    • .ssh/id_rsa
  3. Press return when asks for passphrase.
  4. Press return againg when asks for passphrase.
  5. Copy the public key on the remote machine:
  6. Connect to remote machine:
  7. Append the public key (previously copied from local machine to remote machine) to the list of authorized keys:
    • cat id_rsa.pub >> .ssh/authorized_keys
  8. Generate a public key on the remote machine:
    • ssh-keygen -t rsa
  9. Ok, when asks to save it in: -.ssh/id_rsa
  10. Press return when asks for passphrase.
  11. Press return againg when asks for passphrase.
  12. Copy the public key on the local machine:
  13. Exit from remote server: CTRL+D
  14. Append the public key (previously copied from remote machine into local home) to the list of authorized keys:
    • cat id_rsa.pub >> .ssh/authorized_keys
$\endgroup$
4
  • $\begingroup$ Nice hack :) +1 $\endgroup$
    – sebhofer
    Jan 21, 2013 at 14:19
  • $\begingroup$ Is there a similar solution for the Export? $\endgroup$
    – ybeltukov
    Sep 9, 2013 at 23:16
  • $\begingroup$ @ybeltukov I don't know if it works, but did you try to follow all the steps above and then simply Export["!ssh [email protected] cat /Users/localuser/Desktop/newfile.txt", data, "TSV"], where data is the dataset you want to export? $\endgroup$
    – VLC
    Sep 16, 2013 at 12:13
  • $\begingroup$ @VLC This doesn't work for me with Export["!ssh [email protected] cat /folder/newfile.txt", data] it produces the output : OutputStream["!ssh [email protected] cat /folder/newfile.txt", 236] but actually no newfile.txt is created on my local machine from the remote kernel. $\endgroup$
    – sekisushai
    Jun 16, 2015 at 9:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.