It all started when specifying TargetDevice -> "GPU" in Classify[]
yielded no speedup nor any activity in the GPU. Then I tried something simpler like CUDADot[]
, just to see that it is ~3x time slower than plain Dot[]
:
<< CUDALink`
M = RandomReal[{-1, 1}, {5000, 5000}];
AbsoluteTiming[Dot[M, M];]
(* {2.72066, Null} *)
AbsoluteTiming[CUDADot[M, M];]
(* {8.20455, Null} *)
EDIT: Even after using CUDAMemoryLoad[]
, it still is slower:
<< CUDALink`
M = RandomReal[{-1, 1}, {5000, 5000}];
AbsoluteTiming[Dot[M, M];]
(* {2.29508, Null} *)
CM = CUDAMemoryLoad[M];
AbsoluteTiming[CUDADot[CM, CM];]
(* {7.47153, Null} *)
CUDAMemoryUnload[CM];
After all, perhaps the h/w is to blame, as the same slowdown in Matlab is also seen:
>> x = gpuArray([1:0.3:10000]);
>> tic; dot(x,x); toc
Elapsed time is 0.316340 seconds.
>>
>> x = [1:0.3:10000];
>> tic; dot(x,x); toc
Elapsed time is 0.014257 seconds.
>>
But the workaround (forcing the usage of low-precision floating point arithmetic, i.e. 32 rather than 64 bits) doesn't seem to make things right:
<< CUDALink`
M = RandomReal[{-1, 1}, {5000, 5000}];
AbsoluteTiming[Dot[M, M];]
(* {2.77907, Null} *)
CM = CUDAMemoryLoad[M, "Float"];
AbsoluteTiming[CUDADot[CM, CM];]
(* {8.0693, Null} *)
CUDAMemoryUnload[CM];
My environment, I think, is setup fine:
CUDAQ[]
(* True *)
CUDAInformation[]
(* {1 -> {"Name" -> "GeForce MX150", "Clock Rate" -> 1037500,
"Compute Capabilities" -> 6.1, "GPU Overlap" -> 1,
"Maximum Block Dimensions" -> {1024, 1024, 64},
"Maximum Grid Dimensions" -> {2147483647, 65535, 65535},
"Maximum Threads Per Block" -> 1024,
"Maximum Shared Memory Per Block" -> 49152,
"Total Constant Memory" -> 65536, "Warp Size" -> 32,
"Maximum Pitch" -> 2147483647,
"Maximum Registers Per Block" -> 65536, "Texture Alignment" -> 512,
"Multiprocessor Count" -> 3, "Core Count" -> 96,
"Execution Timeout" -> 1, "Integrated" -> False,
"Can Map Host Memory" -> True, "Compute Mode" -> "Default",
"Texture1D Width" -> 131072, "Texture2D Width" -> 131072,
"Texture2D Height" -> 65536, "Texture3D Width" -> 16384,
"Texture3D Height" -> 16384, "Texture3D Depth" -> 16384,
"Texture2D Array Width" -> 32768,
"Texture2D Array Height" -> 32768,
"Texture2D Array Slices" -> 2048, "Surface Alignment" -> 512,
"Concurrent Kernels" -> True, "ECC Enabled" -> False,
"TCC Enabled" -> False, "Total Memory" -> 2147483648}} *)
CUDADriverVersion[]
(* "430.86" *)
$Version
(* 11.3.0 for Microsoft Windows (64-bit) (March 7, 2018) *)
Any ideas folks? I'd really like to avoid using Matlab ^^. Thanks!
CUDADot
documentation mentionsCUDAMemory
objects, which I would assume to be significantly more efficient to use (data being stored on GPU memory, but also requires explicit memory management). Have you tried toCUDAMemoryLoad
your data, and to timeCUDADot
on these objects only after loading them? $\endgroup$ – kirma Jun 10 '19 at 10:46"Float"
type data, which should correspond with FP32. $\endgroup$ – kirma Jun 10 '19 at 10:56CUDAMemoryLoad[data, "Float"]
. (Note that you have to explicitly free such memory usingCUDAMemoryUnload
to avoid memory leaks.) I don't have a NVIDIA system to try this out at the moment. I speculate thatNumericArray[data, "Real32"]
could work also for your originalCUDADot
example, but I have no way to test this. $\endgroup$ – kirma Jun 10 '19 at 11:26"Float"
type doesn't help! I'm out of ideas. :o $\endgroup$ – kirma Jun 10 '19 at 11:48