12
$\begingroup$

I currently have access to a system with two GPUs: GeForce GTX 1080 and Tesla K40c

I would like to use this machine for training neural networks in Mathematica.

When using the option Target Device->"GPU" for NetTrain it automatically chooses the GTX 1080 GPU for training (I can monitor this from the commandline using nvidia-smi).

However, this card doesn't work with Mathematica yet. It starts training but the loss quickly goes to zero. It's a common problem; see: http://community.wolfram.com/groups/-/m/t/917616

I would like to know how I can fix this issue or configure NetTrain to use the other GPU (Tesla K40c).

More details:

$Version -> 11.0.1 for Linux x86 (64-bit) (September 21, 2016)

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ Target Device->{"GPU",1} $\endgroup$
    – Alexey Golyshev
    Commented Nov 25, 2016 at 6:31
  • $\begingroup$ @AlexeyGolyshev Why don't you write an answer? I wouldn't have found this solution. $\endgroup$
    – Szabolcs
    Commented Nov 25, 2016 at 8:07
  • $\begingroup$ @AlexeyGolyshev Thanks a lot! It worked. Now I will be getting access to a few P100 GPUs soon. Do you know how I can train using multiple GPUs simultaneously? Is it TargetDevice->{"GPU",{1,2,3}} $\endgroup$
    – dan7geo
    Commented Nov 25, 2016 at 8:32
  • $\begingroup$ @dan7geo I don't know. I have only one NVIDIA card. :-) $\endgroup$
    – Alexey Golyshev
    Commented Nov 25, 2016 at 8:49
  • $\begingroup$ @AlexeyGolyshev Guess I'll have to post another question when I do get those cards :) $\endgroup$
    – dan7geo
    Commented Nov 25, 2016 at 9:01

1 Answer 1

Reset to default
9
$\begingroup$

Solution

"TargetDevice"->{"GPU",1}

It should be noted that numeration of GPUs starts from 0.

"TargetDevice"->"GPU" is equivalent to "TargetDevice"->{"GPU",0}

UPDATE

In Mathematica 11.1 numeration of GPUs starts from 1.

Improve this answer
$\endgroup$
3
  • $\begingroup$ @"Alexy Golyshev" What about "TargetDevice"-> {"GPU", All} for multigpu training (which mxnet supports) $\endgroup$
    – user5601
    Commented Mar 6, 2017 at 8:36
  • 2
    $\begingroup$ @user5601 Multi-GPU training support will be in 11.1+ (presentation, slide 19; source) $\endgroup$
    – Alexey Golyshev
    Commented Mar 6, 2017 at 12:15
  • 1
    $\begingroup$ @AlexeyGolyshev You might want to update this answer. Seems like in 11.1 the numbering starts from 1. 'TargetDevice should be either "CPU" or "GPU", or {"GPU", n} for multi-GPU systems, where n is a positive integer that specifies which system GPU to use.' $\endgroup$
    – dan7geo
    Commented Mar 17, 2017 at 0:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.