Why the GPU will cost more time when train the net

I'm in Windows 10

\$Version


11.2.0 for Microsoft Windows (64-bit) (September 11, 2017)

net = LinearLayer[];
AbsoluteTiming[
trained = NetTrain[net, {1 -> 1.9, 2 -> 4.1, 3 -> 6.0, 4 -> 8.1}]]


net = LinearLayer[];
AbsoluteTiming[
trained = NetTrain[net, {1 -> 1.9, 2 -> 4.1, 3 -> 6.0, 4 -> 8.1},TargetDevice -> "GPU"]]


I can reproduce this case,but actually I have a very powerfull gpu as you see

Needs["MXNetLink"];
MXNetLinkPackageScopegetGPUInformation[]


{<|TotalMemory->-2147483648,ComputeCapability->61/10,Name->GeForce GTX1060|>}

Do I have triggered the bug of MMA? I'm in the v11.2. Can anyone reproduce it?

• GPUs are good at performing the same calculations on large arrays. Training a single linear mapping may well be slower on the GPU. If you train networks with millions of parameters, the GPU is much much faster, in my experience. I don't think this is a bug. – Niki Estner Sep 28 '17 at 6:08
• What @nikie says one with caveat: training/inference is only fast if all activation maps and gradients fit in GPU memory. – Sascha Sep 28 '17 at 6:22
• stackoverflow.com/questions/41948406/… – Alexey Golyshev Sep 28 '17 at 8:52
• Actually simply increasing the size of the training data, thus allowing larger batch size, makes the GPU training faster. Try this NetTrain[net, RandomReal[1, 10^5] -> RandomReal[1, 10^5]]`. GPUs are only better than CPUs when a lot of computations can be done in parallel. – dan7geo Sep 28 '17 at 23:59

1 Answer

actually I have a very powerfull gpu as you see

Sorry for the car analogy, but the best way to think if this is: Picture your CPU as a Ferrari race car, and your GPU as a passenger train. If you want to transport one or two people from A to B, the race car is much faster. With the train, you have to get to the station, buy a ticket, wait until the train departs and so on. The race car will easily be 5-10 times faster.

But if you want to transport 1000 people from A to B, the race car will have to make 500 trips back and forth - the train on the other hand can transport 1000 people in one trip. So the train is 50-100 times faster now.

Moral: Only use the GPU if you have large amounts of data that have to be processed with more or less the same code.

• First time to hear such analogy. It is vivid.. – yode Oct 7 '17 at 8:39