Why using compiled code inside compiled code may slow things down?

Question

I made some code that I compiled. The snippet looks as follows:

compiledcode = With[{f1 = function1, f2 = function2, ...}, Hold@Compile[{...},Module[{...},...(*some code involving f1, f2*)...], CompilationTarget -> lang, RuntimeOptions -> "Speed", CompilationOptions -> {"InlineCompiledFunctions" -> True}, RuntimeAttributes ->{Listable}]/.DownValues@.../.OwnValues@...//ReleaseHold]

Here, function1, function2 are some other compiled codes, lang is any. I noticed that if making an alternative code, where I explicitly insert function1, function2 inside compiledcode each time when I need to call them instead of using externally, the code works faster (maybe 2 times or so).

Could you please tell me what can be the reason for this? Unfortunately, I was not able to find a minimal example reproducing the issue, but I think that this may be a kind of a generic issue.

Answer 1

Generic (?) MWE:

The inlined version cf1 copies the argument (CopyTensor) of the inlined function; the equivalent coded cf2 does not have this extra copying operation. (Use CompilePrint to examine.)

cf = Compile[{{x, _Real, 1}}, If[Total[x] > 0, Sin[x], Sin[x]]];
cf1 = With[{nestedCF = cf}, Compile[{{x, _Real, 1}}, nestedCF[x],
    CompilationOptions -> {"InlineCompiledFunctions" -> True}]
   ];

cf2 = Compile[ {{x, _Real, 1}}, If[Total[x] > 0, Sin[x], Sin[x]]];

Needs@"CompiledFunctionTools`"

CompilePrint@cf1

CompilePrint@cf2

Example:

data = RandomReal[{-1, 1}, 10^7];
cf1[data]; // RepeatedTiming
cf2[data]; // RepeatedTiming
(*
{0.0339958, Null}
{0.0277531, Null}
*)

(* cost of CopyTensor: *)
foo = data;
foo[[100]] = 2.; // AbsoluteTiming
(*
{0.00657, Null}
*)

System thresholds can make a big difference:

data = RandomReal[{-1, 1}, 3/2*10^8];
cf1[data]; // RepeatedTiming
cf2[data]; // RepeatedTiming
(*
{0.315479, Null}
{0.255814, Null}
*)

data = RandomReal[{-1, 1}, 2*^8];
cf1[data]; // RepeatedTiming
cf2[data]; // RepeatedTiming
(*
{2.1032, Null}
{0.339828, Null}
*)

As @Henrik pointed out, just how inlined function calls and code are handled and optimized is probably not as generic as supposed in the OP.