Functions vs. patterns

Question

Every time I define a new function, I wonder which construct I should use: true functions obtained by using Function, or rule-based syntax. For example, these are two ways of defining a square function:

square = Function[x, x^2]
(* Alternatively: square = #^2& *)

square2[x_] := x^2

Is there some fundamental difference (e.g. execution efficiency) between the two approaches?

Answer 1

square = Function[x, x^2];
square1 = #^2 &;
square2[x_] := x^2;

DownValues[square]
DownValues[square1]
DownValues[square2]

{}
{}
{HoldPattern[square2[x_]] :> x^2}

Two differences that immediately come to mind are that:

1) functions with down values won't autocompile when you use them in Table, Map, Nest etc. so therefore are less efficient when used that way. e.g.

data = RandomReal[{0, 10}, {10000}];

Timing[Developer`PackedArrayQ[Map[square, data]]]
Timing[Developer`PackedArrayQ[tmp1 = Map[square1, data]]]
Timing[Developer`PackedArrayQ[tmp2 = Map[square2, data]]]

{0.001404, True}
{0.001498, True}
{0.022324, False}

despite tmp1 being packed and tmp2 being unpacked they are equal

tmp1==tmp2
True

but using the pure function gives you a packed list which means faster evaluations and less memory for storage:

N@ByteCount[tmp2]/ByteCount[tmp1]
3.49456

This example used Map but you would observe the same thing with Table, Nest, Fold and so on. As to why this is the case (@Davids question) the only answer I have is the circular one that autocompilation using functions with down values hasn't been implemented. I haven't found out what the difficulties are in implementing this, i.e. whether it hasn't been done because it can't be or because it just hasn't been. Someone else may know and can comment.

2) functions with down values may (in all likelihood will) cause a security warning when present in an embedded CDF.

I'm sure others will be able to expand on this and add many more differences.

Answer 2

These two forms may be similar on the surface, but they are very different in terms of the underlying mechanisms invloved. In a sense, Function represents the only true (but leaky) functional abstraction in Mathematica. Functions based on rules are not really functions at all, they are global versions of replacement rules, which look like function calls.

One big difference is in the semantics of parameter-passing. For rules (and therefore functions based on rules), it is more intruding, in the sense that they don't care about the inner scoping constructs, while Function (with named arguments only) will care. Here is an example of what I mean:

Function[{x},Module[{x=2},x]][3]

(*
  ==> 2
*)

while

In[46]:= 
ClearAll[fn,a];
fn[x_]:=Module[{x=2},{Hold[x],x}]
fn[3]

During evaluation of In[46]:= Module::lvset: Local variable specification 
{3=2} contains 3=2, which is an assignment to 3; only assignments to symbols are allowed. >>

Out[48]= Module[{3=2},{Hold[3],3}]

and especially

In[49]:= 
Clear[a];
fn[a]

Out[50]= {Hold[a$840],2}

Another difference I want to mention is that, due to Function with names arguments beaing a leaky abstraction, passing it as an argument to another function is risky. In addition to the mentioned above post by @WReach, which exposes the essence of the problem, see this post for an example of a trouble that one can get into, in a more "realistic" situation.

That said, in many cases one may think of "functions" defined by rules as normal functions. Because of the powerful builtin rule - dispatching mechanism of rule-application, functions based on rules can be overloaded in very powerful ways, which adds expressive power, as noted by @Sal.

Additionally, while this is not often used, one can often use global pattern-based function definitions as local rules, to achieve rather non-trivial things. In this answer, I use this technique to illustrate how one can dynamically generated Mathematica code at run-time, with the ability to test it in the "interpreter" mode. To illusrate the power of this approach here, I will write a (simplistic, sloppy) macro which will remove the Map auto-compilation limitation mentioned by @Mike Honeychurch, at least for simple pattern-based functions:

ClearAll[withGlobalFunctions];
SetAttributes[withGlobalFunctions, HoldAll];
withGlobalFunctions[code_] :=
  With[{rules =   
       Flatten@Cases[ 
         Names["Global`*"], 
         s_ :> ToExpression[s, InputForm, DownValues]]
       },
    ReleaseHold[
       Hold[code] //. {
         HoldPattern[Map[f_Symbol, args__]] :> Map[f[#] &, args], 
         Sequence @@ rules
       }
    ]
];

Now, taking the same setup:

data = RandomReal[{0, 10}, {500000}];

In[114]:= 
Timing[Developer`PackedArrayQ[Map[square,data]]]
Timing[Developer`PackedArrayQ[tmp1=Map[square1,data]]]
Timing[withGlobalFunctions@Developer`PackedArrayQ[tmp2=Map[square2,data]]]

Out[114]= {0.031,True}
Out[115]= {0.015,True}
Out[116]= {0.032,True}

where what happened is that my macro has expanded the call to square2 before Map was used on it. This is quite non-trivial, becuase it was able to expand the cal square2[#] inside a pure function. This is actually the case of a constructive use of the mentioned above leaky functional abstraction - were Function a complete black box, and this would not be possible. Note that the transformation f->f[#]& is not always correct, since it leaks evaluation. I used it here as an example, but if one were to do it for real, more care must be taken.

Answer 3

Functions are more concise and generally faster but patterns are a lot more expressive. When you don't need the expressive power of patterns you should probably use functions. I use down values more to set up the high level structure of my program and functions to implement the algorithms. But often I am lazy and use down values out of habit. When I am in exploratory mode I pretty much don't worry about the difference.

This example from the Mathematica docs shows one way down values are more expressive (dynamic programming):

fib[n_] :=
 Module[{f},
  f[1] = f[2] = 1;
  f[i_] := f[i] = f[i - 1] + f[i - 2];
  f[n]
  ]

The fact that you can add conditions and the like are another way patterns are more expressive:

f[x_?EvenQ] := 1
f[x_?PrimeQ] := 2
f[x_ /; x > 21 && x < 99] := 3
f[x_] := 4

Of course, you add conditional logic to functions and get a similar effect.

It's also a good idea to use functions when they are side-effect free and down values when there are possible side-effects but this is not something Mathematica enforces and is more of one of my philosophical preferences that comes from exposure to functional programming languages.

I am not sure why down values cause problems in CDF's and I would hope this gets fixed or lessened in future versions. It seems overly restrictive.

Answer 4

The first example (using Function) is called a "pure function" or an "anonymous function". It is used (for example) when you do not want to bother to give a special name to your function. You just describe what it does when applied to its arguments. This is very similar to the pure mathematics

$x\mapsto f(x)$

notation. The emphasis here is on the "abstract" functional aspect (i.e. you think at the function as an object in a functional space).

Typical use is with some kind of options (often from built-in functions) as in:

FixedPoint[(# + 2/# )/2 &, 1`20, SameTest -> (Abs[#1 - #2] < 1*^-10 &)]

where you want to create "on the fly" - without giving a special name - a test function, or in

CellPrint[
 ExpressionCell[x, "Input", 
  CellEvaluationFunction -> (Style[ToExpression[#]^2, Red] &)]]

These are examples from the doc center.

As explained in other answers, its use is less user-friendly, so you will use it (much) less often than the regular

square2[x_] := x^2

style of definition.

Answer 5

I think it is generally a matter of preference.

For my purposes I typically use the second form because it is easier to set up multiple definitions given the same number of inputs and cleaner to perform validation on the inputs. I also find the code easier to understand and debug.

Here is a toy example.

f[x_Integer, y_Integer] := x*y
f[x_List, y_List] := Outer[Times, x, y]
f[___]:= $Failed

Which could be almost equivalently but IMO less cleanly written.

g = Function[{x, y}, 
      Switch[{x, y}, 
        {_Integer, _Integer}, x*y
        , 
        {_List, _List}, Outer[Times, x, y]
        , 
        ___, $Failed]
    ]

I'm at a loss at the moment for how to code that BlankNullSequence bit that would allow g[1] to return $Failed as f[1] would.

One place I often do use the "pure function" style (outside of say graphics options) is when programmatically building up functions.

Here is another toy example that will create a function on the fly that takes 2 or more arguments depending on the inputs to h and when applied to the same number of args, multiplies them. Silly, but you get the idea.

h[arg1_, arg2__] := 
  Block[{x, a},
     x = Table[Unique[a], {Length[{arg1, arg2}]}];
     Hold[Function][x, Times @@ x] // ReleaseHold
  ]

I find this sort of thing more difficult (though not impossible) to do with the f[x_]:= approach.

Answer 6

Preamble

There are some important differences (or, more precisely, features of Function which can't be reproduced with symbols and rules), that have not been reflected in answers here, but that I think deserve a separate answer. These are related to some more advanced uses, involving evaluation control, closures, and garbage collection.

Emulating Hold attributes for SubValue - type definitions

Normally, you can't hold arguments sub__ in a call

fn[args___][sub___]

if your definitions are given as SubValues, like

fn[args___][sub___]:=Hold[args, sub]

because of the way evaluation process work. However, you can define instead

fn[args___]:=Function[##,Hold[args,##],HoldAll]

which would effectively also hold sub. You can't make this work without using a pure function, AFAIK.

Closures and symbol management

Basically, Function is indispensable for creating (nested) closures, and the reason for that is that it spares you from manual symbol management, since there are no explicit names / symbols, to which the action is attached.

It is not very easy to find a good example, because the style of programming based on closures is not very widely used in Mathematica. I will use some very simplified example of what I needed to do at some point. Imagine that you have some data, which you want to split to chunks and present in a form, in which you can apply some transformations to that chunked data in a delayed (lazy) fashion, so that they are only carried out when some specific chunk of data is requested. Here is one way to create such a construct:

ClearAll[lazyListMake]
lazyListMake[data_List, size_] :=
  With[{len = Length[data]},
    LList[
      Function[index, 
        Function[
          Take[data, {
             Min[len, size* (index - 1) + 1], 
             Min[len, size*index]
          }]
        ]]]]

and the data extractor:

ClearAll[getData];
getData[LList[fun_], index_] := fun[index][];

Here is how you can use it:

ll = lazyListMake[Range[30], 5];
getData[ll, 3]

(* {11, 12, 13, 14, 15} *)

The advantage of the nested function construction above is that the inner function serves as a pointer to the data - I can operate on it without extracting the data itself. Now, I can implement a lazy Map operation, for example:

LList /: Map[f_, LList[fun_]] :=
  LList[
    Function[index,
      Composition[
        Function[data, Map[f, data]],
        fun[index]
      ]
    ]]

You can see that this version of Map isn't doing anything when applied, it just transforms the inner function (a "pointer" to actual piece of data), so that Map would actually be applied when we extract the data.

Indeed, we can do:

llm = Map[#^2 + 1 &, ll];
llm1 = Map[Sin, llm];

and no actual work was yet done by Map. When we extract the data, we get:

getData[llm1, 3]

(* {Sin[122], Sin[145], Sin[170], Sin[197], Sin[226]}  *)

and both Map operations only executed now, when we extracted the data, and only on a specific piece of data we wanted.

Returning back to the original question, the above functionality does not need any symbol management, since all functions used here were pure functions. This is a big advantage. This means, in particular, that we can pass expressions involving so constructed lists (LL[...]) anywhere we like, can construct a large number of them, if needed - and in all cases, the destruction of their inner state (when they are no longer used / referenced) is handled automatically by the garbage collector.

Garbage collection of Module - generated variables captured by a closure

Another very interesting, and in fact useful, related feature is that when we leak Module - variables into the closures (creating closures with a state), then those leaked Module variables are garbage-collected once the closure is no longer referenced.

Let's start by executing (on a fresh kernel)

$HistoryLength = 0;

Now consider:

myFun = Module[{y = 1}, Function[var, var + y++]]

(* Function[var$, var$ + y$324++] *)

where we can see that we have now a specific Module - generated variable y$324 as a part of the function's body. Each time we call this function on some argument, that variable gets incremented:

myFun /@ {1, 2, 3}

(* {2, 4, 6} *)

So, we have constructed a closure that was closed over a mutable variable. We can inspect the names:

Names["Global`*"]

(* {"myFun", "var", "var$", "y", "y$324"} *)

to confirm that the variable is in fact visible on the top level. Now, let us call Remove on myFun variable, which stores (references) the functions, and inspect the names of global variables again:

Remove[myFun];
Names["Global`*"]

(* {"var", "var$", "y"} *)

What you see is that the variable y$324 has been garbage - collected. Which is exactly the behavior we want.

The myFun variable was a proxy, which I used to illustrate the mechanism. In practice, it won't be there - we would just pass the pure function directly. So, as soon as it's no longer a part of any expression, the leaked variables will be automatically garbage-collected. Now, should we use named symbols, and that wouldn't be the case, simply because for named symbols, there isn't any automatic garbage collection, so we'd have to do it manually. This means that not only those symbols used for function names would hang around, but also all internal stateful symbols, in such a case.

Summary

I tried to illustrate a few less obvious advantages of having pure functions (Function) in the language, related to the uses of closure and garbage collection.

Answer 7

The second form is usually preferred over the first. This is because it is easier to write functions with multiple parameters (rather than #1, #2, ...), it is more straightforward to associate patterns and conditions on the parameters, and to define functions with variable numbers of parameters.