3
$\begingroup$

I have checked documentation about CUDA usage in Mathematica and it seems that it is quite few applications of CUDA.

Are the any additional packages/functions/tutorials/documentations for CUDA in Mathematica?

I mean some "everyday" calculations:

  1. Working with large lists and calculation of Map, Table, for instance finding list of f[x] for a given list {x}. I know about listability and parallel programming but may be these functions (as Map, Table and similar) can be computed with CUDA. I mean quite "arbitrary" function, not functions from CUDAMap documentation.
  2. In documentation there is no info about zero-frequency component in CUDAFourier. Is this the first element of list as in Fourier?
  3. Can be CUDA used for numerical integration? Do any packages with relates to CUDA and numerical integration exist?
$\endgroup$
3
  • 4
    $\begingroup$ It's all here: reference.wolfram.com/language/CUDALink/guide/CUDALink.html For Map you have CUDAMap and likewise CUDASort, CUDAFold. $\endgroup$
    – flinty
    Commented Jul 2, 2020 at 18:18
  • 2
    $\begingroup$ For CUDAFourier - just read the documentation - it says "The result agrees with the Wolfram Language:" right here so yes it's the same (though consider precision issues as GPU precision may be different from normal Fourier's precision ) $\endgroup$
    – flinty
    Commented Jul 2, 2020 at 21:58
  • $\begingroup$ For 3), that's something I'd like to see Wolfram do more of. CUDA numerical root finding, minimization, integration and monte-carlo, gradients etc... would all add greatly to Mathematica but at the moment we have to write our own kernels. $\endgroup$
    – flinty
    Commented Jul 2, 2020 at 22:20

1 Answer 1

2
$\begingroup$

CUDA (GPU computing) works well as long as you have many independent calculations on lists, matrices,... . Shifting data back and forth from memory CPU to GPU takes time. This spoils the calculation time vs. multi-threaded approaches. In addition, operations that generate e.g. one number out of many (summing up, determinant of matrix,...) take time as many threads wait for the others and a single thread is rather slow. For me this is the reason why GPU applications are still limited.

Memory sharing can overcome this, which is available in the CUDA language. Mathematica has built-in functions to access GPU computations, moreover you can make up your own functions in Mathematica using CUDA libraries. Once you have the framework to link Mathematica functions with CUDA functions, you can do a lot more (e.g. graph theory, ...).

References for CUDA: https://developer.nvidia.com/gpu-accelerated-libraries#deep-learning https://docs.nvidia.com/cuda/#cuda-api-references

$\endgroup$
1
  • 1
    $\begingroup$ I find the answer is a bit limited. I would start of with explaining the difference between CUDA and GPU computing. Then, you pointed to the difficulties that arise due to the limited speed of communication between different memories. But can you support this with small benchmark study? I would also mention cuBLAS as the case where GPU computing is really strong. community.wolfram.com/groups/-/m/t/1128397 $\endgroup$
    – yarchik
    Commented Jul 6, 2020 at 7:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.