12
$\begingroup$

I have spent a lot of time lately training neural networks. However, my poor laptop has only an AMD GPU, so I am stuck training these networks on the CPU. That means I get to train networks over multiple days only to learn that I made a mistake in my loss function.

I don't have enough money to build a new computer or buy an eGPU, so it seems that my best solution is to train these networks on a cloud provider instead.

There are a lot of options nowadays for these cloud GPUS - AWS, Paperspace, Floydhub, etc.

The good news is that MMA neural networks use MXNET under the hood, and you are able to export any network as MXNET files - Export["mynet.json", network] appears to work quite well (though I haven't tried to install MXNET to see if they'll actually work correctly).

Has anybody trained these MMA-defined MXNET neural networks on these cloud providers? If so, what are your recommendations? I am specifically looking for:

  • ease of use - ideally, export->upload->train->download->use in MMA, with limited hassle - I feel that the data is going to be an annoying step here. I'm currently training on 4000 image-mask pairs, a few hundred MB worth of data. Getting some python written to actually train the network is also not going to be very fun.
  • pricing - it seems to be that FloydHub is the best option in terms of price?
  • does anyone know if the Wolfram Cloud is going to support GPU training anytime soon and I can save myself this hassle?
$\endgroup$
  • $\begingroup$ Why not just install WL on an AWS/Paperspace/etc machine and use the NN framework to train directly? (what does exporting to MXNet get you?) $\endgroup$ – Sebastian Sep 3 '18 at 14:38
  • $\begingroup$ I don't think my license allows me to do that, but maybe I'm wrong, I haven't read it in detail. But this would definitely be the easiest solution otherwise... $\endgroup$ – Carl Lange Sep 3 '18 at 18:30
  • $\begingroup$ @CarlLange Please let us know if you succeeded $\endgroup$ – gogoolplex Sep 19 '18 at 12:57
  • $\begingroup$ @gogoolplex I haven't tried yet to be honest. It seems that using a remote kernel on an AWS server is the easiest option but I haven't had the money to spend to do that. I would dearly love to be able to do Device->"Cloud" and not have to deal with any of this, but then transferring datasets etc would be hassle. $\endgroup$ – Carl Lange Sep 19 '18 at 14:19
  • 1
    $\begingroup$ @CarlLange Ok. I know that Google gives you like 300USD for free in the first year and I guess AWS has a similar offer. So lets hope that 12 brings an easy solution to use whatever cloud you want easily. $\endgroup$ – gogoolplex Sep 19 '18 at 17:19
5
$\begingroup$

No need to export to MXNet, put your neural net on a wl script instead and run it on AWS. Install CDFPlayer for linux on a Deep-Learning Ubuntu AMI on a EC2 GPU-compute instance, then download and install manually the drivers and run your training script with TargetDevice->"GPU"

Remember to screen your session to avoid interruption. Check GPU usage with nvidia-smi -l 1

$\endgroup$
  • $\begingroup$ That's really helpful. I didn't realise CDFPlayer would be usable for training my network. I'll try this out over the next few weeks and approve your answer then; in theory looks good! $\endgroup$ – Carl Lange Dec 10 '18 at 15:07
  • $\begingroup$ @CarlLange Yes, the Mathkernel that gets installed with the CDF seems able to do the same things the Mathematica one does; it might be the very same. $\endgroup$ – Fortsaint Dec 10 '18 at 17:40

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.