I am thinking of buying another GPU to accelerate my training. Will I be able to call NetTrain with both GPUs ("TargetDevice"-> {"GPU", All})? Or would it be wasted money?

Best, Max

Edit: What about SLI GPUs? Can I trick Mathematica into thinking its one GPU?

  • $\begingroup$ I don't think NetTrain allows for multi-GPU training at this point in time? Although this could have been implemented in 11.3, when I inquired about this in 11.2 I was told it wasn't possible. $\endgroup$
    – ktm
    Apr 6 '18 at 15:04
  • $\begingroup$ I doubt they train all their pretrained NetModels on only one GPU and if they train them in pure MXNet, some development to do as much as possible in Wolfram should be already in place. $\endgroup$
    – swish
    Apr 6 '18 at 16:02
  • $\begingroup$ I've tested it in Ubuntu 16.04 with Mathematica 11.3, but I encounted a problem about accuracy of training. mathematica.stackexchange.com/q/181942/6648 $\endgroup$ Sep 15 '18 at 17:04

Multi-GPU support very nearly made it for 11.3, and it will definitely be available to users in 12. I will update this answer if it becomes available before that.

Regarding SLI: as far as I know, it has no relevance for CUDA-compute (only graphics rendering for games) and it won't 'trick' NetTrain into considering this as a single GPU.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.