I have to calculate billions of dot products. My current implementation uses Compile to generate C code and it's very fast (10s of seconds). But, I was wondering if I could make it faster by using my GPU (AMD 6670). Unfortunately I don't have an NVIDIA card since it seems that CUDADot is implemented.

I'm trying to use the OpenCL DotProduct code in FileNameJoin[{$OpenCLLinkPath, "SupportFiles", "DotProduct.cl"}]

code below:

src = "/*
 * Copyright 1993-2009 NVIDIA Corporation.  All rights reserved.
 * NVIDIA Corporation and its licensors retain all intellectual property and 
 * proprietary rights in and to this software and related documentation. 
 * Any use, reproduction, disclosure, or distribution of this software 
 * and related documentation without an express license agreement from
 * NVIDIA Corporation is strictly prohibited.

 __kernel void DotProduct (__global float* a, __global float* b, __global float* c, mint iNumElements)
    // find position in global arrays
    int iGID = get_global_id(0);

    // bound check (equivalent to the limit on a 'for' loop for standard/serial C code
    if (iGID >= iNumElements)

    // process 
    int iInOffset = iGID << 2;
    c[iGID] = a[iInOffset] * b[iInOffset] 
               + a[iInOffset + 1] * b[iInOffset + 1]
               + a[iInOffset + 2] * b[iInOffset + 2]
               + a[iInOffset + 3] * b[iInOffset + 3];

I have managed to load the OpenCL function and compute a single DotProdct but this is mostely useless. I've been going over the instructions but I can't seem to figure out how to run many dot products in parallel (the whole point of using the GPU).

srcf = FileNameJoin[{$OpenCLLinkPath, "SupportFiles", "DotProduct.cl"}]

OpenCLDotProduct = 
   "DotProduct", {{"Float"}, {"Float"}, {"Float"}, _Integer}, {16, 

output = {0};
a = {1., 2., 2.};
b = {1., 2., 2.};
OpenCLDotProduct[a, b, output, 3]


output -> {{1., 2., 2.}, {1., 2., 2.}, {9.}}


This is correct, great. However, I can't figure out how to do more than a.b:

output = {0, 0};
a = {1., 2., 2.};
b = {1., 2., 2.};
c = {3., 2., 8.};
OpenCLDotProduct[{a, c}, b, output, 6]


output -> {{{1., 2., 2.}, {3., 2., 8.}}, {1., 2., 2.}, {18., 68.}}


I can't figure out what this output means or what the proper syntax is to do multiple dotproducts. I'm also not sure if this implementation is even fast. I found a blog describing another implementation but I haven't been able to compile it (http://www.openclblog.com/2012/11/opencl-and-dot-product.html).

  • $\begingroup$ If you are using a Mac, be careful: it may be possible to run the opencl code either on the CPU or the GPU and the CPU may be the default. $\endgroup$
    – Szabolcs
    Mar 8, 2013 at 0:39
  • $\begingroup$ @Szabolcs I use both but at the moment I've been working on Win7 x64. My Macbook Air is slower than my PC and it only has the integrated Intel GPU. $\endgroup$
    – s0rce
    Mar 8, 2013 at 0:40
  • 2
    $\begingroup$ The problem is that you can't do this directly using the out-of-the-box solution for a dot product, since GPU-s are directly helpful for massively parallel problems, while this one directly is not. What I would do is to glue all vectors you want to multiple into two large vectors, and pass them to GPU together with a list of lengths. Then, you will have to write your own custom thread scheduler, so that effectively a thread which is finished with one vector pair is rescheduled to another one. Should be doable, but some work. Besides, you need a really powerful GPU for all this to pay off. $\endgroup$ Mar 8, 2013 at 0:52
  • $\begingroup$ As an alternative, you may look for some Map-Reduce frameworks for GPUs, such as this. I never tried those, and it is probably also quite a bit of work. But the advantage here is that problems like your can be treated by Map-Reduce more directly and universally, while the first approach I suggested is not universal. $\endgroup$ Mar 8, 2013 at 0:56
  • $\begingroup$ Also, just to stick with convention, it's best to stick with lower case starting letters for functions. I was confused for a second, thinking, "I didn't know mathematica had an OpenCLDotProduct function already defined!" $\endgroup$
    – user273
    Mar 13, 2013 at 0:08

1 Answer 1


The code is expecting vectors of length 4, and an equal number of a vectors and b vectors.

For example:

n = 10^7;
a = RandomReal[{0, 10}, {n, 4}];
b = RandomReal[{0, 10}, {n, 4}];
output = ConstantArray[0., n];

 rOpenCL = Last @ OpenCLDotProduct[a, b, output, n];]

(*  {1.2324022, Null}  *)

The OpenCL is faster than Mathematica

 rMMA = MapThread[Dot, {a, b}];]

 (*  {15.2568268, Null}  *)

But compiling to C is faster

cf = Compile[{{x, _Real, 2}, {y, _Real, 2}}, MapThread[Dot, {x, y}], 
   CompilationTarget -> "C"];

AbsoluteTiming[rC = cf[a, b];]

(*  {1.0140017, Null}  *)

Also the OpenCL code is less accurate as it uses single precision:

Max[Abs[rOpenCL - rMMA]]
(*  0.0000396708  *)

Max[Abs[rC - rMMA]]
(*  0.  *)

I recompiled the OpenCL function with the actual dot product calculation completely stripped out, and found that the timing barely changed - just copying the data to and from the GPU takes almost all the time here.

  • $\begingroup$ Thanks very much. I guess I'll stick with C. Also if you write the compiled function to be Listable and Parallelized its a bit faster Compile[{{x, _Real, 1}, {y, _Real, 1}}, Dot[x.y], CompilationTarget -> "C", RuntimeAttributes -> {Listable}, Parallelization -> True, RuntimeOptions -> {"Speed"}] $\endgroup$
    – s0rce
    Mar 22, 2013 at 14:51
  • 1
    $\begingroup$ @s0rce, you're welcome. The real benefit of having GPU code for this sort of ultra-simple procedure is when you already have data on the GPU. A while ago I wrote a CUDA kernel to do an element-by-element 2D complex array multiply. As a standalone function it was slower than just doing C = A B in Mathematica, but the overall calculation was an iterative process including 2D FFTs (where the GPU really shines). Having the multiply function in CUDA meant I could load the GPU memory once, do everything on the GPU and then copy the result back to the CPU. $\endgroup$ Mar 22, 2013 at 16:08
  • $\begingroup$ @SimonWoods, Would you mind posting that code if you still have it? I'd like to do something similar with FFTs and multiplying matrices, but I am a newbie to opencl and cuda programming. So, having a working example would be a great help to me. Thanks, Craig $\endgroup$ Apr 21, 2016 at 7:34
  • $\begingroup$ @CraigCarter, sorry but I don't have permission to share that code, I wrote it but I do not own it. $\endgroup$ Apr 21, 2016 at 8:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.