Nonlinear least squares on a GPU with Mathematica 9

Update: A Mathematica wrapper for (https://github.com/zitmen/cuLM) should allow for us to directly implement the Levenberg-Marquardt algorithm in CUDA for nonlinear least squares fitting.

In a previous question of mine (here) I asked how one could best use Mathematica's model fitting capabilities to fit a 2D Gaussian to a set of data.
Two very nice answers were provided by Rahul Narain (who directly computed a mean and covariance matrix for my example data and then used MultinormalDistribution[]) and Sjoerd C. de Vries (who used NonlinearModelFit).

Questions:

• Could these methods be adapted to work on a GPU core in Mathematica 9 (perhaps via CUDALink)?
• Is there some way of using Mathematica to do GPU-based nonlinear least squares (or some other fitting strategy) to accomplish this?
• I found something in www5.cs.fau.de/research/software/cuda-quasi-newton-optimization. I don't know if CUDALink can be used to port these parallel code without much hassle. – PlatoManiac Jun 28 '13 at 8:11
• @PlatoManiac Interesting! – Bob Jun 28 '13 at 8:29
• @PlatoManiac Hmm I'm not so sure this will be easily to implement... though I could certainly be proven wrong. – Bob Jun 28 '13 at 11:03
• Surely it can be done, although I don't have the experience with GPU programming necessary to make any concrete recommendations. I am curious though: I assume you want this because NonlinearModelFit isn't fast enough. But, in (4700), Ajasja and I fitted a very complicated function using the (comparatively inefficient) Nelder-Mead method in not more than a few milliseconds. So, why is it that your fitting of a simple Gaussian is so slow? – Oleksandr R. Jun 28 '13 at 11:33
• @OleksandrR. Though the specific problem may turn out to be too small to reap any actual benefit from the GPU porting of the code I mentioned, it will be really interesting to know how one can port it to Mathematica if possible. This opens up the path for handling large scale optimization problems in Mathematica using CUDA. But we need some C and CUDA specialist here..I do not qualify! – PlatoManiac Jun 28 '13 at 13:35