CUDA and GPU hardware and compatibility questions?

Question

We have iMac desktops and have just begun to parallelize computations across them see: Wolfram Light Weight Grid and parallel computing.

This still does not look like it will give us the performance we would like to see so we've started looking at CUDA.

Unfortunately, Apple informs me that CUDA does not work with the iMac's ATI Radeon HD 4670 GPU cards. Further more, one can't replace the ATI cards with NVIDIA cards on an iMac ;-(

We do have a MacPro and Apple tells us we can replace it's ATI Radeon card with an NVIDIA Quadro 4000 for Mac.

• Has anyone had any successful experience running CUDA functions on this NVIDA Quadro 4000 card?

Apple tells me that the current Mac Pros only support a single graphics card (although the next version of the machine should support multiple ones). This leads to a couple of more questions:

• Could one setup an array of GPU cards in a separate chassis or maybe in a Linux box that Mathematica could access?

• In such a configuration, would one need Mathematica installed on the machine or could it simply see the GPUs as resources on the network?

Just hoping to understand the range of possibilities.

Thanks.

---

P.S. Some more context in response to the first two comments follow.

We have to simultaneously process large sets of sequential data (> 10,000 in length) for survival analysis (related to actuarial studies or time to failure studies). We look at a broad set of inner data structures -- think of them as data compression sets or maybe just a subset of possible Permutations[].

We have developed a custom mixture distribution, which models this data pretty well. Using this custom distribution we find we have four types of calculations that appear to take the most time:

• Fitting parameters for the distribution;
• NExpectation[] at each time step (using the distributions described above);
• NSolve[]to find the intersection of pairs of PDFs of the above distributions; and
• NProbability[]

Currently a single study can take 90+ minutes. We have lots of research to do so anything we can do to speed things up would help a lot. Not certain if a CUDA and GPUs can help on the parameter fitting but, as I understand it, GPUs excel at moving large blocks of data around and floating point calculations.

We have lots of data to move in and out of the processors and wouldn't NExpectation[], NSolve[], and NProbability[] all benefit from using CUDA?

Answer 1

Information we received back from Wolfram Technical Support, which may help others better understand the issues originally raised around CUDA and OpenCL:

In general CUDA is much faster than CPU parallelization CONDITIONALLY and not limited by the limitation of our licenses. However, the functions supported in CUDA are limited to numerical functions such as arithmetic operations and fast Fourier transformation, in particular those defined in math.h in C. You will need to program most of the solvers by yourself or find the corresponding CUDA libraries from either developer zone or C++ extension packages.

The worst drawback of CUDA code is data flow actually. It will take a very long time for data to be copied from RAM to GPU memory. If data flow in/out is what you want, you should do that just once or twice in the whole process.

So far CUDA is only supported on a local machine, i.e. you cannot use the graphics card on other machine. It is not recommended to use two graphics cards even on the same machine so far since it is not mature under current CUDA tech.

On the other hand, parallel cpu computation is less optimized in terms of clock speed. The limitation comes from how CPU generates threads and that the master kernel distributes tasks along the computation. However, the parallel kernels using the gridMAtheamtica tech have excellent scalability and support all internal functions inside Mathematica. This, as a compensation for slower clock speed, saves huge amount of time for programming. Typically for a task involving many different functions like nonlinear solver, plot, symbolic computation, etc.

So here are the options:

1) If you have your own solvers and source codes and you know exactly how to program in terms CUDA C, typically the economical management GPU/CPU memory, CUDA is recommended.

2) otherwise, CPU parallelization is preferred.

Something that might be useful in your code:

a) If you know that the Newtonian method or the like can be applied to solve the intersections, you can either specify that in your NSolve or FindInstance. Another way is to simply write the code and generate CUBIN inside Mathematica.

ref :: CUDALink/tutorial/Programming

b) NProbabilty and NExpectation are gorgeous function since it is functional and symbolic (even though they return numeric value). However, they do take significant CPU seconds to compute and performance is rather hard to improve. The only way is to distribute them on subkernels/other CPU cores.

Note that is not possible to convert functions codes unless there are proper CUDA libraries released.

Last thing about OpenCL and CUDA. CUDA is supposed to be faster than OpenCL and more stable. CUDA definitely need NVIDA card while it is optional to use GPU in OpenCL. But a OpenCL code without calling GPU is a CPU code, which does no good at all.

Answer 2

This is not really an answer to your questions but rather an attempt to provide some extra information that might help you to judge whether GPU will help you.

GPU programming is very effective when it comes to unroll nested loops such as a for loop inside another for loop. The computation inside these loops say a matrix or an image can be readily run in parallel and distributed over the processors on the graphics card. Large sequential data processing might however not gain any performance it might in fact run slower than on the CPU. This is in particular true if large amounts of data have to be loaded and retrieved to and from the GPU many times over the course of the computation as the memory bandwidth from the CPU to the GPU is rather slow. Fast memory is rather scarce at least on the GPUs of standard graphics cards.

I use GPUs to calculate CFD problems which requires solving large matrixes. I experience a performance boost. The loading and unloading of data takes in my situation only a fraction of the overall computation which ranges from 30 minutes to 30 hours.

I can not commend on the Nvidia 4000 card you are considering. My setup are two linux boxes, one with three and one with two GPU units. I have Mathematica installed on one of the machines and I can use the GPUs from Mathematica as expected. I use GPUs on the other machine too but not directly from MMA but rather via a program I have written in CUDA C. I call that program and provide it with parameters from MMA and retrieve the results but I have never tried to use the GPUs directly from within MMA. From all I can see I would need at least a MMA kernel running on that machine to get Mathematica working on it directly. Take this with a grain of salt as I have never tried it!

The best way in my opinion is to analyse your code and see whether and where GPU might be of advantage and then code one of those identified code segments and measure the performance against the standard CPU code.

As comments are restricted in size I have put this into an answer. If this is wrong delete it.