1
$\begingroup$

I am confused about what Compile's Listable and Parallelization options do.

(edit: This post is relevant: stack exchange "The option parallelization for compile")

My initial assumption was that Listable was creating multiple threads on a single core. I have 10 cores on my machine. Here is an example.

Parallelization is True:

  cfParallelizationTrue = Compile[{{x, _Real}}, Sin[Sqrt[Abs[x]]],
      CompilationOptions -> {"ExpressionOptimization" -> True}, 
      RuntimeOptions -> {"Speed", "EvaluateSymbolically" -> False},
      RuntimeAttributes -> {Listable}, Parallelization -> True]

With[{rr = RandomReal[{-1, 1}, 1000]},
 RepeatedTiming[cfParallelizationTrue[rr];]]
(*{0.000055932556, Null}*)

Parallelization is False

cfParallelizationFalse = Compile[{{x, _Real}}, Sin[Sqrt[Abs[x]]],
  CompilationOptions -> {"ExpressionOptimization" -> True}, 
  RuntimeOptions -> {"Speed", "EvaluateSymbolically" -> False},
  RuntimeAttributes -> {Listable}, Parallelization -> False]

With[{rr = RandomReal[{-1, 1}, 1000]},
 RepeatedTiming[cfParallelizationFalse[rr];]]
(*{0.000041559204, Null}*)

Parallelization->False gives about 25% speed up. I don't know why.

Does Parallelization mean that it the computation can use multiple kernels?

LaunchKernels[];
$KernelCount
(*10*)
With[{rr = RandomReal[{-1, 1}, 1000]},
 RepeatedTiming[
  ParallelTable[cfParallelizationTrue[rr];, {i, 1, $KernelCount}]]
 ]
(*0.0042045547*)

If there were no overhead for parallel, this should be about the same as running it once on a single core.

With[{rr = RandomReal[{-1, 1}, 1000]},
 RepeatedTiming[
  ParallelTable[cfParallelizationFalse[rr];, {i, 1, $KernelCount}]]
 ]
(*0.0041659531*)

There seems to be no benefit in any case for using multiple kernels.

Is this because each call to cfParallelizationFalse[rr] is using all my cores because cfParallelizationFalse has Listable->True even though there is only one kernel??

And, cfParallelizationTrue[rr] in a ParallelTable is using all the kernels, but they are running 10 times slower because each Kernel's core is busy ??

$\endgroup$
5
  • 5
    $\begingroup$ Listable means the function is threaded over lists. Compiled functions are different from Regular Listsble functions in that the threading depends on the argument type. A compiled function with a vector argument will thread down to vectors; a regular functions always threads down to scalars $\endgroup$
    – Michael E2
    Aug 6 at 17:10
  • $\begingroup$ Parallelization in the WVM which requires Listable has minimal overhead or at least optimized overhead, I believe. Due to restricted data types and data sharing. $\endgroup$
    – Michael E2
    Aug 6 at 17:20
  • 1
    $\begingroup$ Parallelizing a parallelized process sounds like compressing a compressed file— the result is almost always worse. But I don’t know that about parallelization $\endgroup$
    – Michael E2
    Aug 6 at 17:24
  • $\begingroup$ I think part of my confusion stems from not understanding what a multi-threaded process on a single core does, as opposed to multi-threaded processes on multiple cores. I was under the assumption that each launched kernel was attached to one of the cores—and each of those cores could be multi-threaded. Perhaps Threadable in the WL has nothing to do with running a multi-thread process. $\endgroup$ Aug 6 at 21:14
  • 4
    $\begingroup$ (Thread[]](reference.wolfram.com/language/ref/Thread.html) and process thread are unrelated concepts. Listable is related to the first and not the second. -- Parallelization in the WVM is handled without launching separate kernels, unlike ParallelTable[] and other Parallel* functions. I don't believe kernels are attached to specific cores, but there's usually little point in having more kernels than cores. I think the scheduling of processes is left to the OS, but that's my assumption. $\endgroup$
    – Michael E2
    Aug 6 at 21:57

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.