# Configuring GPU for NetTrain

I currently have access to a system with two GPUs: GeForce GTX 1080 and Tesla K40c

I would like to use this machine for training neural networks in Mathematica.

When using the option Target Device->"GPU" for NetTrain it automatically chooses the GTX 1080 GPU for training (I can monitor this from the commandline using nvidia-smi).

However, this card doesn't work with Mathematica yet. It starts training but the loss quickly goes to zero. It's a common problem; see: http://community.wolfram.com/groups/-/m/t/917616

I would like to know how I can fix this issue or configure NetTrain to use the other GPU (Tesla K40c).

More details:

\$Version -> 11.0.1 for Linux x86 (64-bit) (September 21, 2016)

• Target Device->{"GPU",1} Nov 25, 2016 at 6:31
• @AlexeyGolyshev Why don't you write an answer? I wouldn't have found this solution. Nov 25, 2016 at 8:07
• @AlexeyGolyshev Thanks a lot! It worked. Now I will be getting access to a few P100 GPUs soon. Do you know how I can train using multiple GPUs simultaneously? Is it TargetDevice->{"GPU",{1,2,3}} Nov 25, 2016 at 8:32
• @dan7geo I don't know. I have only one NVIDIA card. :-) Nov 25, 2016 at 8:49
• @AlexeyGolyshev Guess I'll have to post another question when I do get those cards :) Nov 25, 2016 at 9:01

Solution

"TargetDevice"->{"GPU",1}


It should be noted that numeration of GPUs starts from 0.

"TargetDevice"->"GPU" is equivalent to "TargetDevice"->{"GPU",0}

UPDATE

In Mathematica 11.1 numeration of GPUs starts from 1.

• @"Alexy Golyshev" What about "TargetDevice"-> {"GPU", All} for multigpu training (which mxnet supports) Mar 6, 2017 at 8:36
• @user5601 Multi-GPU training support will be in 11.1+ (presentation, slide 19; source) Mar 6, 2017 at 12:15
• @AlexeyGolyshev You might want to update this answer. Seems like in 11.1 the numbering starts from 1. 'TargetDevice should be either "CPU" or "GPU", or {"GPU", n} for multi-GPU systems, where n is a positive integer that specifies which system GPU to use.' Mar 17, 2017 at 0:59