I am thinking of buying another GPU to accelerate my training. Will I be able to call NetTrain with both GPUs ("TargetDevice"-> {"GPU", All})? Or would it be wasted money?

Best, Max

Edit: What about SLI GPUs? Can I trick Mathematica into thinking its one GPU?

  • $\begingroup$ I don't think NetTrain allows for multi-GPU training at this point in time? Although this could have been implemented in 11.3, when I inquired about this in 11.2 I was told it wasn't possible. $\endgroup$
    – ktm
    Commented Apr 6, 2018 at 15:04
  • $\begingroup$ I doubt they train all their pretrained NetModels on only one GPU and if they train them in pure MXNet, some development to do as much as possible in Wolfram should be already in place. $\endgroup$
    – swish
    Commented Apr 6, 2018 at 16:02
  • $\begingroup$ I've tested it in Ubuntu 16.04 with Mathematica 11.3, but I encounted a problem about accuracy of training. mathematica.stackexchange.com/q/181942/6648 $\endgroup$ Commented Sep 15, 2018 at 17:04

1 Answer 1


Multi-GPU support very nearly made it for 11.3, and it will definitely be available to users in 12. I will update this answer if it becomes available before that.

Regarding SLI: as far as I know, it has no relevance for CUDA-compute (only graphics rendering for games) and it won't 'trick' NetTrain into considering this as a single GPU.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.