I usually works remotely using kernels on our hpc and front-end on my local laptop. This enable me do relatively large computation within my laptop(IP address of my laptop may change frequently). But there is a problem when come across packages. Since I manage all the packages on my local computer, I can't just use Get["mypack.m"] when I use remote kernel, instead, I have to copy all the packages to my remote machine before Get. However, I'm constantly developing the packages and copying them back and forth between local and remote computer is quiet tedious. Are there simpler ways to deal with it?
$\begingroup$
$\endgroup$
8
asked Oct 28, 2013 at 17:27
2 Answers
$\begingroup$
$\endgroup$
3
I have not tested this at all, but the default behavior when you Import a package is "Get". If you access your remote machine over the network, isn't an easy method to start an ftp-server on your local machine an load the packages with Import?
For instance it is possible to load packages directly from github (although there, you don't need to login):
Import["https://raw.githubusercontent.com/halirutan/SymbolInformationPalette/master/SymbolInformationPalette.m"]
The same should be possible with ftp, Import and an authentication with username and password. Have you considered this option?
answered Mar 23, 2014 at 10:33
-
$\begingroup$ Thanks, yes I've considered that, but I think that maybe a little bit complicate for me. I never have any experience using a online repository. How do you think the idea in my answer? $\endgroup$ Commented Mar 23, 2014 at 19:10
-
$\begingroup$ My suggestion was not an online repository! I just gave an example that you can load a package with
Importand you are free to use various protocols like http, ftp, sftp. I suggested to load the packages directly from your local machine but with e.g. the ftp protocol. You start your remote kernel from the local machine and the remote machine opens an ftp connection back to your local machine to load the packages. $\endgroup$ Commented Mar 23, 2014 at 20:05 -
$\begingroup$
$\endgroup$
I think the suggestion by rm -rf and halirutan are really great for long term development, but I guess using the online repository is kind of like over shooting the problem for me. My code are relatively small and simple and it would be tedious to push every time I made some small changes.
For my problem, I found out that I can just use the local front end to execute the package on my local computer, then somehow the package will be loaded to the remote kernel:
UsingFrontEnd@NotebookEvaluate["mypack.m"]
answered Mar 23, 2014 at 19:01
git pullbefore aGet(on remote)? $\endgroup$git initin the $UserBase packages directory, but I could be wrong. I'm not certain but I'm willing to be his concern isn't the dropbox/git but correctly setting up package loading on both systems so it works. $\endgroup$$UserBaseDirectoryas a repository, although I think it wouldn't do any harm. I also agree that the OP didn't mention to have concerns about using a repository hoster. I just wanted to make clear that rm-rf's suggestion not necessarily means that this is necessary (in case the OP or any reader would have concerns). Describing such a setup in detail is some work (and to a large extent is covered by the tools documentation) and depends on the OPs specific demands, that's probably why neither me nor anyone else did it... $\endgroup$