10
$\begingroup$

It all started when specifying TargetDevice -> "GPU" in Classify[] yielded no speedup nor any activity in the GPU. Then I tried something simpler like CUDADot[], just to see that it is ~3x time slower than plain Dot[]:

<< CUDALink`
M = RandomReal[{-1, 1}, {5000, 5000}];

AbsoluteTiming[Dot[M, M];]
(* {2.72066, Null} *)

AbsoluteTiming[CUDADot[M, M];]
(* {8.20455, Null} *)

EDIT: Even after using CUDAMemoryLoad[], it still is slower:

<< CUDALink`
M = RandomReal[{-1, 1}, {5000, 5000}];

AbsoluteTiming[Dot[M, M];]
(* {2.29508, Null} *)

CM = CUDAMemoryLoad[M];
AbsoluteTiming[CUDADot[CM, CM];]
(* {7.47153, Null} *)
CUDAMemoryUnload[CM];

After all, perhaps the h/w is to blame, as the same slowdown in Matlab is also seen:

>> x = gpuArray([1:0.3:10000]);
>> tic; dot(x,x); toc
Elapsed time is 0.316340 seconds.
>> 
>> x = [1:0.3:10000];
>> tic; dot(x,x); toc
Elapsed time is 0.014257 seconds.
>> 

But the workaround (forcing the usage of low-precision floating point arithmetic, i.e. 32 rather than 64 bits) doesn't seem to make things right:

<< CUDALink`
M = RandomReal[{-1, 1}, {5000, 5000}];

AbsoluteTiming[Dot[M, M];]
(* {2.77907, Null} *)

CM = CUDAMemoryLoad[M, "Float"];
AbsoluteTiming[CUDADot[CM, CM];]
(* {8.0693, Null} *)
CUDAMemoryUnload[CM];

My environment, I think, is setup fine:

CUDAQ[]
(* True *)

CUDAInformation[]
(* {1 -> {"Name" -> "GeForce MX150", "Clock Rate" -> 1037500, 
"Compute Capabilities" -> 6.1, "GPU Overlap" -> 1, 
"Maximum Block Dimensions" -> {1024, 1024, 64}, 
"Maximum Grid Dimensions" -> {2147483647, 65535, 65535}, 
"Maximum Threads Per Block" -> 1024, 
"Maximum Shared Memory Per Block" -> 49152, 
"Total Constant Memory" -> 65536, "Warp Size" -> 32, 
"Maximum Pitch" -> 2147483647, 
"Maximum Registers Per Block" -> 65536, "Texture Alignment" -> 512,
"Multiprocessor Count" -> 3, "Core Count" -> 96, 
"Execution Timeout" -> 1, "Integrated" -> False, 
"Can Map Host Memory" -> True, "Compute Mode" -> "Default", 
"Texture1D Width" -> 131072, "Texture2D Width" -> 131072, 
"Texture2D Height" -> 65536, "Texture3D Width" -> 16384, 
"Texture3D Height" -> 16384, "Texture3D Depth" -> 16384, 
"Texture2D Array Width" -> 32768, 
"Texture2D Array Height" -> 32768, 
"Texture2D Array Slices" -> 2048, "Surface Alignment" -> 512, 
"Concurrent Kernels" -> True, "ECC Enabled" -> False, 
"TCC Enabled" -> False, "Total Memory" -> 2147483648}} *)

CUDADriverVersion[]
(* "430.86" *)

$Version
(* 11.3.0 for Microsoft Windows (64-bit) (March 7, 2018) *)

Any ideas folks? I'd really like to avoid using Matlab ^^. Thanks!

$\endgroup$
12
  • 1
    $\begingroup$ I'm just curious: does Matlab (which you mention) perform better on the CPU and on the GPU? $\endgroup$ – Szabolcs Jun 10 '19 at 10:34
  • 3
    $\begingroup$ CUDADot documentation mentions CUDAMemory objects, which I would assume to be significantly more efficient to use (data being stored on GPU memory, but also requires explicit memory management). Have you tried to CUDAMemoryLoad your data, and to time CUDADot on these objects only after loading them? $\endgroup$ – kirma Jun 10 '19 at 10:46
  • 4
    $\begingroup$ Actually, this is probably a result of FP64 (64-bit IEEE floating point, the machine precision preferred by Mathematica) arithmetic being quite slow on MX150. Apparently it can perform FP64 at about 35 GFLOPs, while 32-bit floating point (FP32) performs at 1127 GFLOPs. (This is not uncommon on non-compute oriented consumer GPUs.) 35 GFLOPs is well below theoretical performance of a modern AVX2-enabled quad-core CPU. You may be able to speed up your computation by forcing usage of lower-precision "Float" type data, which should correspond with FP32. $\endgroup$ – kirma Jun 10 '19 at 10:56
  • 4
    $\begingroup$ CUDALink tutorial suggests that you could use CUDAMemoryLoad[data, "Float"]. (Note that you have to explicitly free such memory using CUDAMemoryUnload to avoid memory leaks.) I don't have a NVIDIA system to try this out at the moment. I speculate that NumericArray[data, "Real32"] could work also for your original CUDADot example, but I have no way to test this. $\endgroup$ – kirma Jun 10 '19 at 11:26
  • 1
    $\begingroup$ Curious that "Float" type doesn't help! I'm out of ideas. :o $\endgroup$ – kirma Jun 10 '19 at 11:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.