How to define a complicated function inside the body of Compile?

I want to compile a function in a way to keep its memory footprint down. In the example below, I am trying to compile a function f that makes three calls to bigNastyFunction. I do not want to define bigNastyFunction outside and use option "InlineExternalDefinitions" -> True because then three copies of that function will be inserted into the body of the compiled function verbatim, leading to excessive memory usage. Instead, my strategy is to define bigNastyFunction inside a Module which is inside Compile. My hope is that bigNastyFunction will be appropriately stored as a subroutine, and called by the body as needed.

SetSystemOptions["CompileOptions" -> "CompileReportExternal" -> True];

f = Compile[{{x, _Complex}},
Module[{bigNastyFunction = Function[{y}, Sin[y]]},

bigNastyFunction[x] + bigNastyFunction[x^2]^2 + 3 bigNastyFunction[x^3]

]
]


The example above doesn't work because Function is not one of the functions that can be compiled by Compile. What workaround is there to define a large (complicated) function inside the body of a Compile'd function?

I think you need "InlineCompiledFunctions" -> False:

f = With[{bigNastyFunction =
Compile[{{y, _Complex}}, Sin[y](*,CompilationTarget\[Rule]C*)]},
Compile[{{x, _Complex}},
bigNastyFunction[x] + bigNastyFunction[x^2]^2 + 3 bigNastyFunction[x^3],
CompilationOptions -> {"InlineCompiledFunctions" -> False}]]

……
1 C1 = CompiledFunctionCall[ Hold[CompiledFunction[{y}, Sin[y], -CompiledCode-]][ C0]]
2 C2 = Square[ C0]
……


Related post:

Is the CompiledFunctionCall WVM opcode efficient?

CompiledFunctionCall vs. LibraryFunction

• I think this takes more memory and time than just injecting the code (according to ByteCount and RepeatedTiming). – Michael E2 Feb 7 '16 at 3:45
• @MichaelE2 I guess it… depends on how the big nasty function is defined :D – xzczd Feb 7 '16 at 3:51
• If you define bigNastyFunction = Function[{y}, Sin[y]] instead of using Compile, then ByteCount[f] goes down from 11704 to 4560, the same as @MichaelE2 's first answer. – QuantumDot Feb 7 '16 at 11:51
• Your method is always produces functions with about twice as big a ByteCount as mine (more than that when bigNastyFunction is small) because three copies of the compiled function are stored in f. For expressions with a LeafCount up to about 500, your method is up to 20% slower than mine; from 500 or so up to 100,000, they're about the same speed; and at a LeafCount of 180,000, yours was about 10% faster. (The expressions were random plus/times combinations of elementary functions.) – Michael E2 Feb 7 '16 at 13:39

Here is one way:

With[{opts = SystemOptions[]},
With[{bigNastyFunction = Function[{y}, Sin[y]]},
InternalWithLocalSettings[
SetSystemOptions["CompileOptions" -> "CompileReportExternal" -> True],
f = Compile[{{x, _Complex}},
bigNastyFunction[x] + bigNastyFunction[x^2]^2 + 3 bigNastyFunction[x^3]],
SetSystemOptions[opts]
]]]


Here's another way:

Module[{bigNastyFunction = Function[{y}, Sin[y]]},
Block[{x},
f = Compile @@ {{{x, _Complex}},
bigNastyFunction[x] + bigNastyFunction[x^2]^2 + 3 bigNastyFunction[x^3]}
]]


To check:

Needs["CompiledFunctionTools"];
CompilePrint@f

• I think this has little difference with define bigNastyFunction outside and use option "InlineExternalDefinitions" -> True, because then three copies of that function will be inserted into the body of the compiled function verbatim" – xzczd Feb 7 '16 at 3:04
• @xzczd Depends on how the big nasty function is defined.... – Michael E2 Feb 7 '16 at 3:08