Automating SFTP upload within Mathematica?

Question

I have MMA-generated html and would like to automate uploading to a named directory to a web host via SFTP or HTTP POST, ideally without creating local temp files. The closest question I could find on MMA.SE is this question which doesn't really address this issue.

Based on this "minimal http upload gci" in Python, and HTTP Request-Authentication flow-chart, it seems that uploading by FTP may be relatively easier. Sjoerd C. de Vries suggested this question may answer the POST part.

Others have found FTP in non-interactive mode to be difficult. But one would think authentication, put and closing the session in interactive mode could be managed within MMA.

However, Run is not meant to capture output from the external program. From the sparse documentation, I think RunThrough is meant to capture external program output, but I can only manage to capture an exit condition code:

RunThrough["sftp", "####@understars.org:temp.understars.org"]

Doesn't output to the front end, but does set the Out[] to 256.

So how can an interactive FTP session be automated?

(Note I've tagged this "html" as well since ultimately that is what I want to export.)

Answer 1

Using a slightly modified version of vngx-jsch (source included), an open-source implementation of jsch, and JLink and a small but efficient Mathematica package this is now easily possible. All code can be browsed here, and most simply be installed by executing this twitterable line: (tested on Windows, Linux and Mac, but not on the Raspberry Pi). It should all work on Mathematica 7, 8, 9 and 10 (though I did not test the new version on Mathematica 7 and 8).

Get @ "http://goo.gl/y4pCZ8"

This loads from Github my updated MathematicaPackageInstall function in order to automatically install the package from Github to your local $UserBaseDirectory/Applications folder. At the end of the installation process (give it half a minute) a notebook MSFTP.nb opens with some examples in it.

If you have set up your sftp (ssh) server (any Linux box will do) and opened port 22 on your firewall you can do things like

MSFTPPut[NotebookDirectory[], "test", "Password"->"secret",
  "UserName" -> "testuser", "HostName" -> "testserver.example.com"]]

There is also a MSFTPGet function for getting files and directories.

As a goodie I added PassEncode, so you can encode your password on your $MachineID, i.e., on my machine

PassEncode["secret"]

gives

   {40, 42, 33, 49, 72, 33, 42, 41, 57, 50, 125, 10, 64, 120, 122, 106, \
124, 63, 34, 77, 113, 55, 94, 57, 113, 110, 89, 70}

which can be used insted of "secret" in MSFTPPut or MSFTPGet. So the advantage is you don't have to save clear-text passwords in your notebook, but some $MachineID dependent encryption of it.