# How to write complex function definitions at run time?

(Posting as a self-answered question to share a technique I discovered)

In a certain project, I have many function definitions in which all parameters should be optional and come with default values. A simplified example is the following:

ClearAll[sampleFunction];
Options[sampleFunction] = {
"a" -> 1000,
"b" -> {},
"c" -> 42};
sampleFunction[OptionsPattern[]] :=
Module[{
a = OptionValue["a"],
b = OptionValue["b"],
c = OptionValue["c"]},
Append[b, a + c]];

sampleFunction[]

{1042}

sampleFunction["b" -> {1, 2, 3}, "a" -> -38]

{1, 2, 3, 4}


The problem with this pattern is there is a lot of repetitive boilerplate, increasing the risk of typographical errors and fatigue for readers and writers of the code (imagine that the option names are long and complicated and that there are dozens of them, not just three as in the example).

I'd like to have a syntax like the following,

functionator[
sampleFunction,
<|"a" -> 1000, "b" -> {}, "c" -> 42|>,
Append[b, a + c]]


that minimally captures the essential features of the definition without undue repetition.

functionator must transform its second argument, an Association of option names and default values, into a Module, but it's not straightforward. For instance, the following attempt just produces syntax errors in the Module specification because things are all being evaluated at inappropriate times:

ClearAll[functionator];
SetAttributes[functionator, HoldAllComplete];
functionator[nym_, args_, body_] :=
With[{
keys = Keys[args],
vars = Symbol /@ Keys[args],
vals = Values[args]},
ClearAll[nym];
Options[nym] = Normal[args];
nym[OptionsPattern[]] :=
Module[
{var, key} \[Function] var = OptionValue[key],
{vars, keys}
], body]];

functionator[
sampleFunction,
<|"a" -> 1000, "b" -> {}, "c" -> 42|>,
Append[b, a + c]]

sampleFunction[]

During evaluation of In[317]:= Module::lvlist: Local variable specification
MapThread[Function[{var$,key$},var$=OptionValue[sampleFunction,{},key$]],
{{a,b,c},Keys[Association[a->1000,b->{},c->42]]}] is not a List. >>

Module[
Function[{var$, key$},
var$= OptionValue[sampleFunction, {}, key$]], {{a, b, c},
Keys[Association["a" -> 1000, "b" -> {}, "c" -> 42]]}],
Append[b, a + c]]


Trying to sort this out with various Hold..., Release, and Evaluate combinations was exhausting and not converging.

There must be a better way.

-

This question is closely related to:

In my answer there I gave a couple of abstractions to simplify definitions of the type you describe. I shall reiterate my approach with adjustment for your syntax.

## Code using listWith

SetAttributes[{listWith, defWithOpts2}, HoldAll]

listWith[(set : Set | SetDelayed)[L_, R_], body_] :=
set @@@ Thread[Hold @@@ {L, R}, Hold] /. _[x__] :> With[{x}, body]

defWithOpts2[h_Symbol, asc_Association, RHS_] :=
With[{keys = Keys[asc]},
With[{syms = Join @@ ToHeldExpression[keys]},
Options[h] = Normal[asc];
h[OptionsPattern[]] := listWith[syms = OptionValue[keys], RHS]
]
]


Example:

defWithOpts2[
sampleFunction,
<|"a" -> 1000, "b" -> {}, "c" -> 42|>,
Append[b, a + c]
]

sampleFunction["b" -> foo[bar], "a" -> 17]

foo[bar, 59]


This generates a definition that is reliant upon listWith:

sampleFunction[OptionsPattern[]] :=
listWith[Hold[a, b, c] = OptionValue[{"a", "b", "c"}], Append[b, a + c]]


I actually prefer this as I think the syntax is cleaner, e.g. there is only one OptionValue call.

## Code without listWith

If you dislike the listWith dependency here is the function with the expansion included:

SetAttributes[defWithOpts3, HoldAll]

defWithOpts3[h_Symbol, asc_Association, RHS_] :=
With[{keys = Keys[asc]},
With[{syms = Join @@ ToHeldExpression[keys]},
Options[h] = Normal[asc];
Inner[Set, syms, OptionValue /@ Hold @@ keys, Hold] /. _[x__] :>
(h[OptionsPattern[]] := With[{x}, RHS])
]
]


The definition it creates:

sampleFunction[OptionsPattern[]] :=
With[{a = OptionValue["a"], b = OptionValue["b"], c = OptionValue["c"]},
Append[b, a + c]]

-
The following works cleanly: ClearAll[sampleFunction2]; defWithOpts2[ sampleFunction2, <|"size" -> 10|>, size]; sampleFunction2[] but the following works, but with ugly warnings :( ClearAll[sampleFunction3]; defWithOpts2[ sampleFunction3, <|"size" -> 10|>, Module[{xs = ConstantArray[Null, size]}, xs]]; sampleFunction3[] – Reb.Cabin Oct 19 '14 at 0:19
@Reb I either don't understand or I am not seeing the problem here. From sampleFunction3[] I get {Null, Null, Null, Null, Null, Null, Null, Null, Null, Null} with no messages issued. – Mr.Wizard Oct 19 '14 at 1:43
@MrW yeah, I am still investigating; I am definitely getting messages ConstantArray::ilsmn: "Single or list of non-negative machine-sized integers expected at position 2 of ConstantArray[Null,size]. " and Join::normal: "Nonatomic expression expected at position 1 in Join[size]. " in MMA 10.0.1.0. I do get the correct result. – Reb.Cabin Oct 19 '14 at 2:04
@Reb That almost sounds like HoldAll is not set on listWith. Would you mind copying the entire contents of a Notebook evaluated in a fresh kernel that exhibits this behavior? – Mr.Wizard Oct 19 '14 at 2:24
Ok, it was a stray ClearAll on defWithOpts2. I have a habit of writing ClearAll and SetAttributes right above every definition, and I had one too many :( – Reb.Cabin Oct 19 '14 at 15:53

The new cluster of Inactive, Inactivate, and Activate heads in 10.0+ were designed for exactly this task. So far as I can tell, they give approximately the same level of functionality as macros in Common Lisp.

Consider the following preliminary step that exhibits the desired rewrite in inactive form, just to show the technique at work

ClearAll[functionatorExpander];
SetAttributes[functionatorExpander, HoldAllComplete];
functionatorExpander[nym_, args_, body_] :=
With[{
vars = Symbol /@ Keys[args],
vals = Values[args]},
ClearAll[nym];
Options[nym] = Normal[args];
Inactive[SetDelayed][nym[OptionsPattern[]],
Inactive[Module][
{var, key} \[Function]
Inactive[Set][
var,
OptionValue[key]],
{vars, Keys[args]}
], Inactivate[body]]]];

functionatorExpander[
sampleFunction,
<|"a" -> 1000, "b" -> {}, "c" -> 42|>,
Append[b, a + c]] // StandardForm


Here is another version that activates the generated code, with a couple of unit tests

ClearAll[functionator];
SetAttributes[functionator, HoldAllComplete];
functionator[nym_, args_, body_] :=
With[{
vars = Symbol /@ Keys[args],
vals = Values[args]},
ClearAll[nym];
Options[nym] = Normal[args];
SetDelayed[nym[OptionsPattern[]],
Activate[
Inactive[Module][
{var, key} \[Function]
Inactive[Set][
var,
OptionValue[key]],
{vars, Keys[args]}
], Inactivate[body]]]]];

functionator[
sampleFunction,
<|"a" -> 1000, "b" -> {}, "c" -> 42|>,
Append[b, a + c]]

sampleFunction[]

{1042}

sampleFunction["b" -> {1, 2, 3}, "a" -> -38]

{1, 2, 3, 4}

-
FYI: you have an evaluation leak e.g. setting b = "Fail!"; beforehand results in "Module::lvset: Local variable specification {a=1000,Fail!={},c=42} contains Fail!={}, which is an assignment to Fail!; only assignments to symbols are allowed. >>" edit: Actually setting b = "Fail!"; afterward also causes the generated definition to fail because Module is inactive at the time of RHS evaluation. This is unacceptable IMHO. – Mr.Wizard Oct 16 '14 at 19:09
After some (re)consideration I am down-voting this answer in its present condition. Even if the leak mentioned were acceptable for this particular problem the inactivation of Module essentially breaks Module and makes this construct largely pointless. Sorry, but -1 :-( – Mr.Wizard Oct 16 '14 at 19:36
The new constructs you mentioned are really not in the same league as macros in CL, simply because in Mathematica there is no read time (at least not in the sense in which it is present in CL), and everything is happening at run-time. Therefore, you can't safely assume that the code (except macros) is truly inert, and so far I haven't seen a compelling case where the new constructs would significantly alleviate the pain of evaluation control. – Leonid Shifrin Oct 16 '14 at 19:52
@Leonid Interesting that you appear now; do you have notification set up on the tag meta-programming? If not perhaps you should as you are surely the master of it. :-) – Mr.Wizard Oct 16 '14 at 19:58
@Mr.Wizard I saw this question the moment it was asked. Unfortunately, I don't have the time at the moment for anything more than a few comments. But, chances are that I wouldn't produce a better answer here than you did. If I had the time, I'd try to address a problem more generally, but that would require a lot more time than I currently have. – Leonid Shifrin Oct 16 '14 at 20:07