5
$\begingroup$

Is there any option to directly import to a Windows-driven Mathematica kernel a file from a remote server without copying it on hard drive? Right now I'm using CopyFile with RemoteFile and $SSHAuthentication. It is very inconvenient because for every remote file I have to provide a path on harddrive to copy it and then use just regular Import. It there a workaround?

Import["/server/folder", "Username" -> "username", "Password" -> "password"] from Import files from password-protected remote server doesn't work, as well as Import[RemoteFile[...]]

Improve this question
$\endgroup$
2
  • $\begingroup$ Does the plan you accepted apply to you? $\endgroup$
    – yode
    Commented Feb 29 at 10:05
  • $\begingroup$ Yes, the edited answear works - You can use SSHFS-Manager (or sshfs command) to mount a remote drive and then just regular import within mathematica works, but it is considerably slower than just copy file to local hard drive via WinSCP and importing it from local drive to Mathematica. $\endgroup$
    – Lechuu
    Commented Mar 2 at 10:12

2 Answers 2

Reset to default
6
$\begingroup$

Since your know ssh, why not directly use linux commands like

 Import["!echo `ssh user@host 'cat  /home/user/file.txt'`", "String"]

You can avoid password request by simply sharing public/private key with your computer and server (see ssh-keygen and ssh-copy-id -i ~/.ssh/mykey user@host in ubuntu)

Edit 1 Other solution (not related to Mathematica) is to use sshfs command to mount remote server folder to local computer file system. Seems there are attempts to implement the functionality for windows as well. See, https://stackoverflow.com/questions/67182778/mounting-linux-fs-in-windows-10-using-sshfs for instructions. After mounting remote folder effectively becomes a local folder, so every program can access files as if they were located on local computer.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thanks for Your answer!I'm not familiar with linux commands since unfortunately I work on windows. I'm actually using ssh key pair but this approach doesn't work - it just imports the string in quotes. maybe I have to provide somehow a path to a private key before importing? $\endgroup$
    – Lechuu
    Commented Apr 27, 2023 at 15:42
  • $\begingroup$ First, check if you can connect to the remote server without password. You can read this hostinger.com/tutorials/how-to-use-putty-ssh . In the end they have a section how to generate public key for putty. I think in windows the example will not work anyway, because the string "!echo ...." is in fact is linux terminal command. I suspect it is possible to write something similar in powershell, but I don't know it. $\endgroup$
    – Acus
    Commented Apr 27, 2023 at 15:57
  • $\begingroup$ Oh, I'm sorry, I'm using putty with ssh key and I can connect without password - it works great (via WinSCP works as well). I don't really know how to connet from powershell, but when I find it i will try to use it in Mathematica. Thanks! $\endgroup$
    – Lechuu
    Commented Apr 27, 2023 at 16:22
  • 1
    $\begingroup$ @Lechuu Another solution which comes to mind is to use sshfs command to mount remote server folder to local computer file system. I use this routinely and I just checked that recently there were attempts to do this for windows also Look for example at stackoverflow.com/questions/67182778/… . After mounting remote folder it effectively becomes as a local folder, so every program can access files as if they were located on local computer. Sometimes this is really convenient. $\endgroup$
    – Acus
    Commented Apr 28, 2023 at 7:53
  • $\begingroup$ Thank You very much! It is exactly what I was looking for! If you could just edit Your answer so everyone could know the solution I would be very greateful. $\endgroup$
    – Lechuu
    Commented Apr 28, 2023 at 12:23
2
$\begingroup$

I'm in Windows 11. If you are currently on a LAN, then I recommend using this scheme:

ImportString[URLRead["sftp://12.12.12.57/home/user/42.jpg", 
  Authentication -> <|"Username" -> "user", "Password" -> "yourpass"|>]]

Or directly:

ImportString[URLFetch["sftp://12.12.12.57/home/user/42.jpg","Username" -> "user", "Password" -> "yourpass"]]
URLExecute["sftp://12.12.12.57/home/user/42.jpg",Authentication -> <|"Username" -> "user", "Password" -> "yourpass"|>]

1.)Or use the newest method in Acus' answer:

Import["!ssh -i \"C:/Users/myname/.ssh/key\" [email protected] cat /home/user/42.jpg","JPEG"]

2.)Or we can use RemoteEvaluate+$DefaultRemoteKernel(Personal most recommended):

RemoteEvaluate[KernelConfiguration[
   "ssh://[email protected]/usr/local/bin/wolfram", 
   "SshIdentity" -> "C:/Users/myname/.ssh/pycharmkey2"], Import["42.jpg"]]

But note the kernel object describes a method for launching a kernel; it does not represent a persistent connection. We can get a persistent connection like following:

ker = KernelConfiguration[
   "ssh://[email protected]/usr/local/bin/wolfram", 
   "SshIdentity" -> "C:/Users/myname/.ssh/pycharmkey2"];
launchker = First[LaunchKernels[ker]];

ParallelEvaluate[Import["42.jpg"], launchker]

3.)Or use RemoteRunProcess+RemoteConnect(Fastest):

ImportString[
 RemoteRunProcess["12.12.12.57" , "cat 42.jpg", "StandardOutput", 
  Authentication -> <|"Username" -> "user", "Password" -> "yourpass"|>]]

4.)Or use wolframscript:

ImportString[RemoteRunProcess["12.12.12.57", 
"/usr/local/Wolfram/Mathematica/13.3/Executables/wolframscript \
-code 'Import[\"/home/user/42.jpg\"]' -format JPEG", "StandardOutput",
   Authentication -> <|"Username" -> "user", "Password" -> "yourpass"|>]]

However, wolframscript starts an mma kernel each time it runs, which is slow unless you use wolframscript -wstpserver. But use wolframscript -wstpserver, need to advance in the server startup SystemFiles/Links/wstpserver/wstpserver. But if you try to use the wstpserver to transfer large files, its efficiency is catastrophically slow.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Very interesting, so many solutions. However some uses plain password input, which is consider as very unsafe. On linux we never display it, therefore I would prefer solutions with SshIdentity. $\endgroup$
    – Acus
    Commented Mar 12 at 10:37
  • $\begingroup$ @Acus The input method of the password is not important, basically all methods can use the password and key two ways $\endgroup$
    – yode
    Commented Mar 12 at 18:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.