Using Compile Function Options efficiently

Question

While this gives general pointers, I want inputs on ground rules for improved performance of nested/interdependent compiled functions.

sample problem: a, b, c, and d are four supporting compiled functions that interact among themselves and also support the main function main, as defined below:

a = Compile[{{p1, _Real}, {p2, _Real}}, Min[p1, p2]];
b = Compile[{{p1, _Real}, {p2, _Real}}, Max[p1, p2]];
c = Compile[{{p1, _Real}, {p2, _Real}}, Min[1, 0.5 + a[p1, p2]/b[p1, p2]], 
       CompilationOptions -> {"InlineExternalDefinitions" -> True}, 
    RuntimeAttributes -> {Listable},Parallelization -> True];
d = Compile[{{p1, _Real}, {p2, _Real}, {p3, _Real}}, a[Mod[p1, 5] - p2, p3], 
       CompilationOptions -> {"InlineExternalDefinitions" -> True}, 
    RuntimeAttributes -> {Listable}, Parallelization -> True];

main = Compile[{{aa, _Real, 0}}, Module[{
       k1 = RandomReal[10, {10^4, 5}], 
       k2 = RandomReal[5, {10^4, 5}], 
       k3 = RandomReal[15, {10^4, 5}]}, 
       aa + b[k1, k2] + c[k2, k3] + d[k1, k2, k3]], 
       CompilationOptions -> {"InlineExternalDefinitions" -> True}];

Do[main[10],{100}];//AbsoluteTiming

I found that eliminating redundant options for dependent functions can help improve the performance; e.g. Since a and b are used in c and c is defined as listable and parallelized, 1) a and b need not be defined listable and defining them as listable may deteriorate the performance of c (using AbsoluteTiming) 2) since a and b are not defined listable, they can be inlined in c that will further improve performance, compared to using "InlineExternalDefinitions" -> True alone.

With this understanding I want inputs to help establish ground rules that will enhance the performance of main:

1. Which functions need CompilationOptions -> {"ExpressionOptimization" -> True}, RuntimeOptions -> "Speed" and why not others?
2. Which functions need CompilationTarger-> "C" and why not others?
3. Since main calls b, c and d and already has RuntimeAttributes -> {Listable}, Parallelization -> True, none of the supporting functions should need to be listable, but it will not work. Why?
4. What else can be done to improve the performance of main (I need to use a compiled function in NMinimizer) ?

Answer 1

1. Optimising the code given

So I've completely rewritten my answer to this optimization, after @brama pointed out my versions returned a scalar rather than a matrix - my mistake, apologies.

First recall the performance of @brama's code:

Do[main[10], {100}]; // AbsoluteTiming
(* 1.10 seconds *)

But what if we compile @brama's main function to C, like I've done? Well, on my computer it takes about the same amount of time as it does compiled to MVM.

So here's my version:

main2 = Compile[{{aa, _Real, 0}}, 
    Module[{
     k1 = RandomReal[10, {10^4, 5}],
     k2 = RandomReal[5, {10^4, 5}],
     k3 = RandomReal[15, {10^4, 5}]},
    10 + Max @@ {k1, k2}
       + Map[Min[1., #] &, 
          0.5 + MapThread[Min, {k2, k3}, 2]/MapThread[Max, {k2, k3}, 2], {-1}]
       + MapThread[Min, {(Mod[#, 5] & /@ k1) - k2, k3}, 2]
    ], 
    CompilationTarget -> "C"
   ];

Do[main2[10], {100}]; // AbsoluteTiming
(* 0.37 seconds *)

(* Check it outputs a matrix, not a scalar *)
main2[10]

(*{{18.8678, 22.1486, 20.0878, 20.6308, 17.7006},
 ...9998...,
 {22.5947, 19.6328, 22.6912, 19.3589, 18.1205}}*)

Nearly a 3x speed-up!

One last sanity check (although all the functions I've used are in here List of compilable functions)

Needs["CompiledFunctionTools`"]
CompilePrint[main2]
(* No calls to MainEvaluate, no instances of CopyTensor *)

So it's a good compilation. By constrast, the original code in @brama's main returns a few instances of CompiledFunctionCall, which could impact on performance.

Comparison with the uncompiled version

What happens if you don't compile to C, but to MVM instead? Well my version takes about 1.2 seconds, so that isn't great. And what about the MMA-only version?

uncompiledMain[num_] := Module[{
   k1 = RandomReal[10, {10^4, 5}],
   k2 = RandomReal[5, {10^4, 5}],
   k3 = RandomReal[15, {10^4, 5}]},
  num + Max @@ {k1, k2} + 
   Map[Min[1., #] &, 
    0.5 + MapThread[Min, {k2, k3}, 2]/
      MapThread[Max, {k2, k3}, 2], {-1}] +
   MapThread[Min, {(Mod[#, 5] & /@ k1) - k2, k3}, 2]
  ];

Do[uncompiledmain[10], {100}]; // AbsoluteTiming
(* 11.51 seconds *)

Considerably slower. So compilation all the way to C is useful for my implementation, but not useful for yours.

Larger matrices

What about with larger matrices {k1, k2, k3}?

(* Update definitions of k1, k2 and k3 to *)
{
 k1 = RandomReal[10, {10^7, 5}],
 k2 = RandomReal[5, {10^7, 5}],
 k3 = RandomReal[15, {10^7, 5}]
}

main[10]; // AbsoluteTiming
(* 5.58 seconds *)

main2[10]; // AbsoluteTiming
(* 4.75 seconds *)

So the speed-up is a bit smaller here. I wonder why?

Further analysis

Interesting things happen when we take {k1,k2,k3} out of the compiled function.

bramasMain = Compile[{{aa, _Real, 0}, {k1, _Real, 2}, {k2, _Real,2}, {k3, _Real, 2}}, 
    Module[{},aa + b[k1, k2] + c[k2, k3] + d[k1, k2, k3]],
   CompilationOptions -> {"InlineExternalDefinitions" -> True}
  ];

blochwavesMain = Compile[{{aa, _Real, 0}, {k1, _Real, 2}, {k2, _Real, 2}, {k3, _Real,2}}, 
Module[{}, 10 + Max @@ {k1, k2} + 
    Map[Min[1., #] &, 
     0.5 + MapThread[Min, {k2, k3}, 2]/
       MapThread[Max, {k2, k3}, 2], {-1}] +
    MapThread[Min, {(Mod[#, 5] & /@ k1) - k2, k3}, 2]
   ], 
   CompilationTarget -> "C"
  ];

First, it allows us to check the two functions are equivalent, as both receive the same set of random numbers.

AbsoluteTiming[
 externalk1 = RandomReal[10, {10^7, 5}];
 externalk2 = RandomReal[5, {10^7, 5}];
 externalk3 = RandomReal[15, {10^7, 5}];
]
(* 1.59 seconds *)

AbsoluteTiming[result1 = bramasMain[10, externalk1, externalk2, externalk3];]
(* 4.29 seconds *)

AbsoluteTiming[result2 = blochwavesMain[10, externalk1, externalk2, externalk3];]
(* 3.24 seconds *)

result1 == result2
(* True *)

Norm[result1 - result2]
(* 0. *)

What about if we compile bramasMain to C?

bramasMainC = Compile[{{aa, _Real, 0}, {k1, _Real, 2}, {k2, _Real,2}, {k3, _Real, 2}}, 
    Module[{},aa + b[k1, k2] + c[k2, k3] + d[k1, k2, k3]],
   CompilationOptions -> {"InlineExternalDefinitions" -> True},
   CompilationTarget = "C"
  ];

AbsoluteTiming[result3 = bramasMainC[10, externalk1, externalk2, externalk3];]
(* 5.01 seconds *)

Compiling to C is now slower! Why is this? It's likely that the CompiledFunctionCalls found when you do CompilePrint[bramasMainC] are indeed affecting the performance, because the functions a, b, c and d aren't inlined. It would seem to me that when compiling only to MVM rather than C, the overhead with the CompiledFunctionCall is less.

---

2. Your questions

Returning to your questions in the original post, these were:

1. Which functions need CompilationOptions -> {"ExpressionOptimization" -> True}, RuntimeOptions -> "Speed"} and why not others?
2. Which functions need CompilationTarget -> "C" and why not others?
3. Since main calls b, c and d and already has RuntimeAttributes -> Listable}, Parallelization -> True, none of the supporting functions should need to be listable, but it will not work. Why?
4. What else can be done to improve the performance of main (I need to use a compiled function in NMinimize)?

Q1. I'll attempt to answer this.

Regarding ExpressionOptimization, from the documentation

If the compiled function needs to make an external call to the Wolfram Language, the default setting (for ExpressionOptimization) of Automatic does not optimize

In an ideal world, you want to minimize any external calls in a compiled function. Therefore, if you have no external calls, the default setting is sufficient. Otherwise you need to set it to True and then compare the performance with the default setting to see if it is beneficial.

Regarding RuntimeOptions -> "Speed", again look at the documentation, you need to consider what this option actually means, and if you are putting speed before quality in a critical section of your code (e.g. catching overflow). Focus on this first before speed.

Q2. & Q3. Hopefully someone might chip in here.

Regarding the instruction to compile to C, this is complex. Notice that in my analysis above, when I compiled to C and found your function ran slower, I only had "CompilationTarget" -> "C" inside the main function, not in c or d. If you compile c and d to C as well, there is a modest speed-up, but (about 20% or so), but because the functions are inlined, there is still the problem with the CompiledFunctionCalls.

Q4. Is this actually the function you want to use?

If it isn't (and main is in fact more complex), my suggestion would be to optimize it as far as you are able, and if the performance is still not what you want, post a question here with your actual function asking for pointers about optimization. For example, compiling to C might be beneficial there, and that confuses matters when it comes to your actual problem, since as we've seen compiling to C doesn't always improve performance.

This makes sense in response to your statement:

I want inputs on ground rules for improved performance of nested/interdependent compiled functions.

As the sample problem shows, it's not quite so simple. I've managed to turn your nested functions into a single compilable function with a sometimes significant speed-up, with no need for listable behaviour in the compilation options.

This may well be the case with your real problem. So focus on your actual function and problem-at-hand, and remember the community is here to help!