3
$\begingroup$

I'm in Windows 10

$Version

11.2.0 for Microsoft Windows (64-bit) (September 11, 2017)


net = LinearLayer[];
AbsoluteTiming[
 trained = NetTrain[net, {1 -> 1.9, 2 -> 4.1, 3 -> 6.0, 4 -> 8.1}]]

net = LinearLayer[];
AbsoluteTiming[
 trained = NetTrain[net, {1 -> 1.9, 2 -> 4.1, 3 -> 6.0, 4 -> 8.1},TargetDevice -> "GPU"]]

I can reproduce this case,but actually I have a very powerfull gpu as you see

Needs["MXNetLink`"];
MXNetLink`PackageScope`getGPUInformation[]

{<|TotalMemory->-2147483648,ComputeCapability->61/10,Name->GeForce GTX1060|>}

Do I have triggered the bug of MMA? I'm in the v11.2. Can anyone reproduce it?

$\endgroup$
  • 1
    $\begingroup$ GPUs are good at performing the same calculations on large arrays. Training a single linear mapping may well be slower on the GPU. If you train networks with millions of parameters, the GPU is much much faster, in my experience. I don't think this is a bug. $\endgroup$ – Niki Estner Sep 28 '17 at 6:08
  • $\begingroup$ What @nikie says one with caveat: training/inference is only fast if all activation maps and gradients fit in GPU memory. $\endgroup$ – Sascha Sep 28 '17 at 6:22
  • $\begingroup$ stackoverflow.com/questions/41948406/… $\endgroup$ – Alexey Golyshev Sep 28 '17 at 8:52
  • $\begingroup$ Actually simply increasing the size of the training data, thus allowing larger batch size, makes the GPU training faster. Try this NetTrain[net, RandomReal[1, 10^5] -> RandomReal[1, 10^5]]. GPUs are only better than CPUs when a lot of computations can be done in parallel. $\endgroup$ – dan7geo Sep 28 '17 at 23:59
5
$\begingroup$

actually I have a very powerfull gpu as you see

Sorry for the car analogy, but the best way to think if this is: Picture your CPU as a Ferrari race car, and your GPU as a passenger train. If you want to transport one or two people from A to B, the race car is much faster. With the train, you have to get to the station, buy a ticket, wait until the train departs and so on. The race car will easily be 5-10 times faster.

But if you want to transport 1000 people from A to B, the race car will have to make 500 trips back and forth - the train on the other hand can transport 1000 people in one trip. So the train is 50-100 times faster now.

Moral: Only use the GPU if you have large amounts of data that have to be processed with more or less the same code.

$\endgroup$
  • $\begingroup$ First time to hear such analogy. It is vivid.. $\endgroup$ – yode Oct 7 '17 at 8:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.