6
$\begingroup$

Sometimes it might be useful to override Mathematica's built-in functions with custom code that is either more specific or more efficient (or even less "smart"), but at the same time keeping the same calling syntax.

For example, I found that for my purposes the built-in Expectation operator in Mathematica is very, very, very inefficient in computing the expectation of a function based on my own custom distribution. However, I can manually integrate it (i.e. just use brute force sum) very efficiently.

Since sometimes I switch back and forth between my own custom distribution and the built-in distributions of Mathematica, I would very much like to maintain the form Expectation[ expr, {x1,x2,x3} \[Distributed] dist ], where expr is some expression that involves the random variables x1,x2,x3 and also some non-random variables y1,y2,y3 that I can substitute in arbitrary non-random numbers into.

So suppose we have two distributions, say mathdist that's any of the Mathematica built-in distribution where if dist is mathdist I want to use the built-in Expectation computation method; and say I define and fixate the name mydist of which when dist is that, I will compute the expectation via myExpectation[expr_].

Question: How can I modify Expectation so that it can differentiate these two and then use the appropriate integration / summation method?

$\endgroup$
  • $\begingroup$ Unless you are a very expert Mathematica user, I wouldn't recommend modifying existing functions as there might be other functions affected that depend on Expectation. Could you not write your own function to call myExpectation[expr_] when desired and Expectation otherwise? $\endgroup$ – JimB Sep 28 '15 at 4:45
  • $\begingroup$ Thanks. I actually thought about using Unprotect to modify the definition and indeed, had submitted that as my answer to my own question. But upon testing that method further --- even if I work it inside a Module, hoping that variable scoping would limit the effects of Unprotect (it didn't) --- my approach was behaving very unexpectedly. In the end, I ended up doing something what you'd suggested. Thanks for the thought! $\endgroup$ – user32416 Sep 28 '15 at 5:38
  • 1
    $\begingroup$ Have a look at this package I wrote: library.wolfram.com/infocenter/MathSource/3336 . The notebook "Basic overriding" should give some advice on how to... override Mathematica functions. (I also had an 'override package' in the works but then mma's version changed... $\endgroup$ – Peltio Sep 28 '15 at 14:02
  • $\begingroup$ @Peltio Thank you so much! I'll definitely have a good look! $\endgroup$ – user32416 Sep 28 '15 at 17:06
  • $\begingroup$ Could you give us an example where Expectation is inefficient? If you can give a relatively clear example we might be able to improve the built-in methods! $\endgroup$ – Stefan R Sep 28 '15 at 20:02
5
$\begingroup$

So, I had a look at that old version of my notebook and I found it too specific. I copy here part of what was the documentation of my Override.m package. It's more general. (I'll spare you the code required to add options...)

Here we go.

It's easy to redefine a built-in procedure so that it can call itself without incurring in an infinite recursion. The key idea is to condition the new definition by means of variable, say $override, that prevents built-in evaluation and diverts the call to user defined code. If this code needs to call the built-in version of the procedure, all it has to do is to change the value of the variable $override so that it won't call itself. In a way the use of this variable is analogous to specifying the prefix 'super' that is used in Java programming to refer to a method of a superclass from within an overriding method with the same signature. Here, the value of $override is initialized to True in order to prevent Mathematica from taking over the user defined code. Protecting the variable is a good idea.

$override = True;
    Protect[$override];

Now, whenever we want to redefine a built-in procedure, let's call it BuiltinProc, in term of itself we just define it with a conditioned left hand side (so that the definition applies only when $override = True) and enclose the portion of code that calls the original procedure in a Block that sets $override = False. Better yet we can call, from inside the Block, our own code that can make use of the built-in procedure as we would in a fresh Mathematica kernel. Let's call userProc the user defined code:

Unprotect[BuiltinProc];
BuiltinProc[argsandopts___ ] /; $override == True :=
        Block[
          {$override = False},
      userProc[argsandopts]
      ];
Protect[BuiltinProc];

(Unprotecting and Protecting are needed since the built-in symbols are by default protected). Note that when the argument sequence argsandopts is the same as for the built-in procedure (as is the case with such a general pattern matching scheme) we are in fact redefining it. Once we've done this, we can write the code for userProc and allow it to call the built-in procedure at will: since it's called inside a Block that sets $override=False we do not need to worry about the value of $override inside the code. Calling BuiltinProc in the global environment (where $override is True) will result in a call to userProc and in a (possible) call to BuiltinProc inside a Block where $override is False. There is no danger to incur in an infinite recursion since the two calls to BuiltinProc are in fact different and complementary. This is a sample code that adds to the previous snippets and shows that whenever userProc is called it will use the built-in procedure whose arguments and options have been suitably processed before being passed to it:

userProc[args___, opts___Rule] :=
  (
    Print["This code was called from a Block 
    that temporarily set $override to  ", $override, ". \nThe following \
function contains a call to the original BuiltinProc without conditions." ];
    oldargs = FA[args]; oldopts = FO[opts];
    myFun[
        BuiltinProc[oldargs, oldopts],
         otherargsandopts
      ]
    )

When we call BuiltinProc in a global environment where $override=True we are in fact calling userProc that, in turn, can use the built-in code of the original built-in procedure:

BuiltinProc[x, y, {a, b}, myopt -> Automatic]

This code was called from a Block that temporarily set $override to False. The following function contains a call to the original BuiltinProc without conditions.

myFun(BuiltinProc(FA(x, y, {a, b}), FO(myopt -> Automatic)), otherargsandopts)

Example: Fourier rescaled

Let's try it out. We redefine Fourier in order to divide its result by the square root of the length of the list passed to it. This is the result of the built-in procedure Fourier in a fresh Mathematica kernel:

Fourier[{1, 2, 3, 4}] // Chop

{5., -1. - 1. I, -1., -1. + 1. I}

If we redefine the procedure in a way that it can call itself:

$override = True;

Unprotect[Fourier];
Fourier[data_List ] /; $override == True := Block[{$override = False},
      Fourier[data]/Sqrt[Length[data]]
      ];
Protect[Fourier];

We can have a new version of Fourier that works according to our taste (on version 4 we should consider the option for FourierParameters, while on previous versions the new definition can be used to add exactly that option).

Fourier[{1, 2, 3, 4}] // Chop

{2.5, -0.5 - 0.5 I, -0.5, -0.5 + 0.5 I}

Tah dah!

To go back to 'factory settings' we just clear our definition

Unprotect[Fourier]; Clear[Fourier]; Protect[Fourier];

The following conditional assignment allows for another kind of unsetting, since it is possible to use the form of the left hand side to choose the assignment we did (this one come from Villegas' tutorial on working with unevaluated expressions - I wish I had read it before creating my packages)

Unprotect[Fourier];
Unset[Fourier[data_List] /; $override == True];
Protect[Fourier];
$\endgroup$
  • $\begingroup$ Oh wow! This is a great answer! And it'll definitely take me some time to fully digest your great work! Thanks! And I also love the fact that it's general enough that I can use it in other contexts! $\endgroup$ – user32416 Sep 28 '15 at 23:29
  • $\begingroup$ Yes, with your permission I could also change the title and wording of your question to "Overriding a built-in function with my own procedure". $\endgroup$ – Peltio Sep 28 '15 at 23:31
  • $\begingroup$ Yes, please do! And also feel free to modify my question so that it fits your general context. Thanks again! $\endgroup$ – user32416 Sep 28 '15 at 23:32
  • $\begingroup$ You're welcome. When you have coded your solution (I guess you'll have to add a test inside the block to decide when to use brute force or mma's built-in Expectation), you might want to answer your own question to show how you solved the inefficiency problem. $\endgroup$ – Peltio Sep 28 '15 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.