Let's say I have a list of random numbers;

list = Table[Random[], {100000}];

I want to apply a function f in every element of the list and take their sum. An obvious solution to that would be


For the sake of clarity I am using here the following simple f:

f=#^2&;SetAttributes[f, Listable]

(Of course, the function Power is Listable by default


outputs 0.158 on my machine.)

But as always I want the fastest routine I can get. So I used Compile to reduce the running time:


Testing with


I got a worse result, namely 0.288! :-(

I tested again putting f by hand inside the compiled function:

Do[fcfast[list], {100}] // AbsoluteTiming

Output was 0.054 and I was pleased!

Why is this happening? How can I speed up my routines without having to put the function explicitly inside Compile?

Sjoerd C. de Vries answered on the question Using Apply inside Compile tracing with Needs["CompiledFunctionTools"] and the function CompilePrint that explains the timing differences but I still don't know how to improve that.

Results after applying the best solution :

I am adding here my test results using for various implementations. I added CompilationOptions -> {"InlineExternalDefinitions" -> True} which answered my question.

The list i used contained 1000000 random numbers , f=#^2& and the compiler used was the MinGW. enter image description here

  • $\begingroup$ @Sjoerd C. de Vries thank you for the edit. I re-edited to add a relevant question you had answered. $\endgroup$
    – tchronis
    Commented Nov 3, 2013 at 19:03
  • $\begingroup$ Well, thanks for referring to my answer. However, I have some difficulties seeing the connection. It's about Goto and Compile and does not deal with timing. $\endgroup$ Commented Nov 3, 2013 at 19:15
  • $\begingroup$ When running CompilePrint for the two compiled functions i got differences in one line. fc gave T(R1)1 = MainEvaluate[ Hold[f][ T(R1)0]] and fcfast gave T(R1)1 = Square[ T(R1)0]. $\endgroup$
    – tchronis
    Commented Nov 3, 2013 at 19:23
  • $\begingroup$ Related $\endgroup$ Commented Nov 4, 2013 at 9:12
  • 1
    $\begingroup$ it doesn't make a difference to the suggested solution, but I think it should be noted that f=#^2&;SetAttributes[f, Listable] does not achieve what you expect, the attributes of f don't influence how the pure function is evaluated. You woul instead want f=Function[Null,#^2,Listable]. Of course, as you mentioned, here it doesn't make a difference since Power is listable anyway. You can see the difference when using any function that isn't listable instead of Power... $\endgroup$ Commented Nov 4, 2013 at 11:48

1 Answer 1


Why is this happening? How can I speed up my routines without having to put the function explicitly inside Compile?

It is happening because Compile has the attribute HoldAll


(* {HoldAll, Protected} *)

This means, that no evaluation of the arguments will happen. In your case the arguments to your Compile call are {{list,_Real,1}} and Total[f[list]]]. While you think that your definition of f is put inside there, exactly this does not happen. It will stay unevaluated as you can see in the output of CompilePrint

<< CompiledFunctionTools`
CompilePrint[Compile[{{list, _Real, 1}}, Total[f[list]]]]

1   T(R1)1 = MainEvaluate[ Hold[f][ T(R1)0]]
2   R0 = TotalAll[ T(R1)1, I0]]
3   Return
" *)

Therefore, your compiled function cannot calculate the Total directly, it has to ask the Kernel every time the function is called. This results directly in a slowdown. A really easy solution to your problem is to use With because it replaces every occurrence of a named value with it's value. I didn't write variable on purpose here, because in With you don't define variables, instead it is more like constant defines. Although looking a bit awkward, this

fc = With[{f = f},
  Compile[{{list, _Real, 1}}, Total[f[list]]]

is one solution of your problem. What happens is that the named identifier f on the left hand side of the = in the With call is connected to the value of your global f. Inside the body of the With, every occurrence of f is replaced by this value, just like you would copy and paste it manually.

Another way for injection would be

fc = Function[func, Compile[{{list, _Real, 1}}, Total[func[list]]]][f]

which has the short form

fc = Compile[{{list, _Real, 1}}, Total[#[list]]] &[f]

Or you can even use Replace in combination with Hold.

In the end, you should follow Sjoerd's suggestion and use CompilePrint to check whether you compiled function works how you expect it.


Simon reminded me that there is another simple solution, which is the "InlineExternalDefinitions" options:

Compile[{{list, _Real, 1}}, Total[f[list]], 
 CompilationOptions -> {"InlineExternalDefinitions" -> True}]
  • $\begingroup$ thank you very much. At last i can compile my functions the way i want. One note : New timings were much closer to that of Compile[{{list,_Real,1}},Total[list^2]]. In fact they were just x2.I used in both "With" and "&[f]" the CompilePrint and noticed that the only difference with the explicitly defined one was one extra line at the top T(R1)1 = CopyTensor[ T(R1)0]]. I have no problem with this overhead i am just wondering if it is absolutely necessary. $\endgroup$
    – tchronis
    Commented Nov 4, 2013 at 8:02
  • 1
    $\begingroup$ I think it would be good to mention the option CompilationOptions -> {"InlineExternalDefinitions" -> True} as well. $\endgroup$ Commented Nov 4, 2013 at 9:13
  • $\begingroup$ @SimonWoods Thanks, I forgot about this one. $\endgroup$
    – halirutan
    Commented Nov 4, 2013 at 9:24
  • $\begingroup$ Thanks @SimonWoods and Halirutan , the CompilationOptions -> {"InlineExternalDefinitions" -> True} is the most transparent and efficient of all. $\endgroup$
    – tchronis
    Commented Nov 4, 2013 at 17:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.