Making a symbol's new definitions be tried before all previously defined ones

Question

Is there any way to "close" a package (or a symbol, or a context) in that if a user of the package adds definitions to the symbol they will be tried before the package defined ones, just like what happens with built-ins?

Answer 1

I will suggest a solution for DownValues - based definitions, but it may be generalized to other types of definitions as well. I will only consider a case of a single symbol, but again, the generalization is rather straightforward. You will also have to execute your code in a special dynamic environment.

A first ingredient of my suggestion is a (slightly modified) symbol-cloning functionality described here:

Clear[GlobalProperties];
GlobalProperties[] := 
 {OwnValues, DownValues, SubValues, UpValues, 
    NValues, FormatValues, Options, DefaultValues, Attributes};

ClearAll[clone];
Attributes[clone] = {HoldAll};
clone[s_Symbol, new_Symbol] :=
  With[{clone = new, sopts = Options[Unevaluated[s]]},
    With[{setProp = (#[clone] = (#[s] /. HoldPattern[s] :> clone)) &},
      Map[setProp, DeleteCases[GlobalProperties[], Options]];
      If[sopts =!= {}, Options[clone] = (sopts /. HoldPattern[s] :> clone)];
      HoldPattern[s] :> clone]]

Here comes my suggested dynamic environment then:

ClearAll[withUserDefs];
SetAttributes[withUserDefs, HoldAll];
withUserDefs[sym_Symbol, {defs__}, code_] :=
  Module[{s, inSym},
    clone[sym, s];
    Block[{sym},
     defs;
     sym[args___] /; ! TrueQ[inSym] :=
        Block[{sym, inSym = True},
          clone[s, sym];
          With[{result = sym[args]},
             result /; result =!= Unevaluated[sym[args]]
          ]
        ];
     code]];

What is happening here: first, we clone the original symbol. Then, we Block it, and run the definitions which we want to override the previous ones. Then, we add a catch-all definition which uses the Villegas-Gayley trick, but also Block-s the symbol in question again, and inside the inner Block uses the clone to effectively "unblock" the symbol to its original defs by reverse-cloning it, and run those. The extra trick to use With and a shared local variable result is needed to avoid infinite iteration in cases when neither user-defined nor previous rules apply.

Here comes an example:

ClearAll[f];
f[1] = 10;
f[x_] := x^2;
f[x_, y_] := x + y;

And now:

withUserDefs[f, {f[x_] := x^4}, {f[1], f[2], f[1, 2], f[1, 2, 3]}]

(* 
  ==> {1, 16, 3, f[1, 2, 3]}
*)

you can see that in the first two cases, the modified definitions were used, in the third one the original definition was used, and the last one did not match any and evaluated to itself.

One can nest these constructs, and the inner one will override the outer ones then. For example:

withUserDefs[f, {f[x_] := x^4}, 
   withUserDefs[f, {f[1] := 100}, 
     {f[1], f[2], f[1, 2], f[1, 2, 3]}]]

(* 
  ==> {100, 16, 3, f[1, 2, 3]}
*)

You can ask why I wasn't just using the Villegas-Gayley trick by itself, which is much simpler. The answer is that there is no guarantee that the ordering of definitions will be right with it, even if we manually reorder them, and moreover, cases like f[1] = 10 are immune to all reorderings of DownValues and will always be at the top, so can not be dealt with at all, within a pure Villegas-Gayley approach - but can be successfully dealt with in this more complex one.

The suggested approach is as good as the symbol's cloning procedure is. For SubValues, UpValues etc, it should be modified. The full solution involving all possible global rules and multiple symbols will likely look more complex, but I just wanted to illustrate the idea in the simplest possible setting. Also, we probably can not count on such generalization to be fully robust.

Answer 2

If we are the ones writing the package in question, then we could proceed as follows. First, we define a public version of the function that delegates all calls to a private version:

ClearAll[publicFn]
publicFn[args___] := privateFn[args]

Then we define the functionality we desire on the private function:

ClearAll[privateFn]
privateFn[x_] /; x < 0 := 100 x
privateFn[x_] := 10 x

Before users make any modifications, the public function behaves in the way we have defined:

publicFn /@ Range[-5, 5]

{-500,-400,-300,-200,-100,0,10,20,30,40,50}

Now, if a user comes along and adds a definition to the public function...

publicFn[n_?EvenQ] := 0

... then the corresponding behaviour is overridden, but the other definitions remain in place.

publicFn /@ Range[-5, 5]

{-500,0,-300,0,-100,0,10,0,30,0,50}

Since the public function is defined with the broadest possible definition, all more-specific user definitions will be tried before the pre-defined rules. Of course, if the user redefines the public function's one definition exactly, then all bets are off.

Handling Partial Functions

If the function is only partially defined, we may want the public function to return unevaluated when the arguments are outside its domain. To do this, we need a slightly more elaborate version of the public function:

ClearAll[publicFn]
publicFn[args___] := Module[{v = privateFn[args]}, v /; !MatchQ[v, _privateFn]]

Unfortunately, Mathematica no longer considers this definition to be "broad" due to the complex MatchQ condition. Consequently, this definition will no longer be applied after subsequent user-supplied definitions. What we need to do is take over the management of the down-values of publicFn so that our "broad" definition is always last in the list. This is accomplished by installing a new SetDelayed/Set definition as an up-value on publicFn:

publicFn /: (SetDelayed|Set)[h_publicFn, def_] :=
  (DownValues[publicFn] =
    Insert[
      DeleteCases[DownValues[publicFn], Verbatim@HoldPattern[h] :> _]
    , HoldPattern[h] :> def
    , -2
    ];)

We'll change privateFn so that it is only partially defined:

ClearAll[privateFn]
privateFn[x_] /; x < 0 := 100 x

publicFn /@ Range[-5, 5]

{-500, -400, -300, -200, -100, publicFn[0], publicFn[1], publicFn[2], publicFn[3], publicFn[4], publicFn[5]}

Users can then add further partial definitions:

publicFn[n_?EvenQ] := 0
publicFn[5] = "will be replaced by a subsequent definition";
publicFn[5] = "five";

{-500, 0, -300, 0, -100, 0, publicFn[1], 0, publicFn[3], 0, "five"}

Answer 3

Here is method based on WReach's answer. It works by explicitly sorting the list of DownValues after any new assignment is made, placing the ___ last. It allows the return the public function unevaluated. By nature this only works for DownValues definitions.

ClearAll[func, privateFn]

privateFn[x_] /; x < 0 := 100 x
privateFn[x_?NumericQ] := 10 x

func[args___] := With[{eval = privateFn[args]}, eval /; ! MatchQ[eval, _privateFn]]

func /: (set : Set | SetDelayed)[lhs_func, rhs_] /; ! TrueQ[$deffunc] :=
  Block[{$deffunc = True},
    lhs ~set~ rhs;
    DownValues[func] = 
     SortBy[
       DownValues[func],
       {MatchQ[ #[[1]], _[_[_[_, Verbatim[___]]]] ] &}
     ];
  ]

Testing:

func[n_Integer?EvenQ] := 0
func[n_Integer] := 1

func /@ Range[-5, 5]
func[0.7]
func["undefined"]

{1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1}

7.

func["undefined"]

Answer 4

Ok, based on your answers...

ClearAll[publicFn]
Module[{guard = True},
 publicFn[args___] /; guard := 
  Block[{guard = False}, Module[{res = publicFn[args]},
    If[MatchQ[res, _publicFn],
     res = privateFn[args]]; res /; ! MatchQ[res, _privateFn]]
   ]
 ]

This doesn't suffer problems of the automatic DownValue resorting, and without resorting to upvalues on Set functions... Shouldn't have problems with UpValues either. The public definition of the function is found perhaps before or perhaps after some user defined definitions. But if it is found, it makes sure there aren't any other good public definitions before transferring the job to the private definition. If there are none, it returns unevaluated.

privateFn[x_] /; x < 0 := 100 x
privateFn[x_?NumericQ] := 10 x
privateFn[3] = 99

publicFn /@ Range[-5 , 5]
publicFn["oij"]


{-500, 0, -300, 0, -100, 0, 10, 0, 30, 0, 50}  
publicFn["oij"]

Now, making some definitions

publicFn[n_?EvenQ] := 0;
publicFn[3 | 33] = -100;
soh /: publicFn[soh] := "boo"

We see that

publicFn /@ Range[-5 , 5]
publicFn["oij"]
publicFn[soh]

returns

{-500, 0, -300, 0, -100, 0, 10, 0, -100, 0, 50}

publicFn["oij"]

"boo"