Do people actually use UpValues?

Question

I know what they are and how to define them. They have to serve some purpose - else why include them? But I never used them and all examples on this site and the docs never show a practical use of UpValues.

Can someone give me an example to see when they are actually needed?

Answer 1

Yes, UpValues are certainly useful in that you can bind definitions and custom behaviours to the symbol rather than the operator. For instance, I can define (a simple, silly example):

g /: g[x_] + g[y_] := g[x] g[y]

to actually multiply the two when I add them. This definition is now stored in:

UpValues[g]
(* {HoldPattern[g[x_] + g[y_]] :> g[x] g[y]} *)

The alternative would be to unprotect Plus and then overload it with this definition as:

Unprotect@Plus;
g[x_] + g[y_] := g[x] g[y]
Protect@Plus;

The advantages of Upvalues over overloading built-in operators are:

• It is safer. Modifying built-ins is risky because you don't know what might break internally with your custom definitions
• Mathematica reads custom definitions first (except for perhaps Times) before built-in ones. As a result, overloading operators and functions with more and more additional definitions could slow things down because it has to consider the custom definitions even in situations where they aren't necessary.
• All the custom definitions for a symbol/object are collected in its UpValues. With the alternate approach, you don't know, at a glance, which functions have been modified to treat this symbol/object differently.

---

For a more informative (and relevant) example, I'll turn to Sal Mangano's "Mathematica Cookbook", Chapter 2: Functional Programming, which illustrates the idea and reinforces the points I made above:

There are some situations in which you would like to give new meaning to functions native to Mathematica. These situations arise when you introduce new types of objects. For example, imagine Mathematica did not already have a package that supported quaternions (a kind of noncommutative generalization of complex numbers) and you wanted to develop your own. Clearly you would want to use standard mathematical notation, but this would amount to defining new downvalues for the built-in Mathematica functions Plus, Times, etc.

Unprotect[Plus,Times]
Plus[quaternion[a1_,b1_,c1_,d1_], quaternion[a2_,b2_,c2_,d2_]] := ...
Times[quaternion[a1_,b1_,c1_,d1_], quaternion[a2_,b2_,c2_,d2_]] := ...
Protect[Plus,Times]

If quaternion math were very common, this might be a valid approach. However, Mathematica provides a convenient way to associate the definitions of these operations with the quaternion rather than with the operations. These associations are called UpValues, and there are two syntax variations for defining them. The first uses operations called UpSet (^=) and UpSetDelayed (^:=), which are analogous to Set (=) and SetDelayed (:=) but create upvalues rather than downvalues.

Plus[quaternion[a1_,b1_,c1_,d1_], quaternion[a2_,b2_,c2_,d2_]] ^:=  ...
Times[quaternion[a1_,b1_,c1_,d1_], quaternion[a2_,b2_,c2_,d2_]] ^:= ...

The alternate syntax is a bit more verbose but is useful in situations in which the symbol the upvalue should be associated with is ambiguous. For example, imagine you want to define addition of a complex number and a quaternion. You can use TagSet or TagSetDelayed to indicate that the operation is an upvalue for quaternion rather than Complex.

quaternion /: Plus[Complex[r_, im_], quaternion[a1_,b1_,c1_,d1_]] := ...
quaternion /: Times[Complex[r_, im_], quaternion[a1_,b1_,c1_,d1_]] := ...

Upvalues solve two problems. First, they eliminate the need to unprotect native Mathematica symbols. Second, they avoid bogging down Mathematica by forcing it to consider custom definitions every time it encounters common functions like Plus and Times. (Mathematica aways uses custom definitions before built-in ones.) By associating the operations with the new types (in this case quaternion), Mathematica need only consider these operations in expression where quaternion appears. If both upvalues and downvalues are present, upvalues have precedence, but this is something you should avoid.

Answer 2

Oh yes. UpValues are used quite a bit. There are several common uses, and you may have a look at this and especially this question and answers therein to see some sample uses.

As for practical uses: I will just mention a couple of examples for what I consider to be the main practical use: overloading functions (system or user-defined) on custom data types, so that such redefinitions are local (in the sense that they are attached to the heads representing the new data type, rather than to the functions being overloaded).

One example is my implementation of the large data framework, where it would not be an exaggeration to say that UpValues were crucial element of it. I routinely and automatically create thousands of them during the operation of the framework, and they serve as a powerful encapsulation mechanism, which allowed me to use the OOP-style encapsulation in a very effective way. I will reproduce here the main function using them:

ClearAll[definePartAPI];
definePartAPI[s_Symbol, part_Integer, dir_String] :=
 LetL[{sym = Unique[], hash = Hash[sym], 
     fname = $fileNameFunction[dir, hash]
   },
   sym := sym =  $uncompressFunction@$importFunction[fname];
   s /: HoldPattern[Part[s, part]] := sym;

   (* Release memory and renew for next reuse *)
   s /: releasePart[s, part] :=
       Replace[Hold[$uncompressFunction@$importFunction[fname]], 
          Hold[def_] :> (ClearAll[sym]; sym := sym = def)];

   (* Check if on disk *)
   s /: savedOnDisk[s, part] := FileExistsQ[fname];

   (* remove from disk *)
   s /: removePartOnDisk[s, part] := DeleteFile[fname];

   (* save new on disk *)
   s /: savePartOnDisk[s, part, value_] :=
      $exportFunction[fname, $compressFunction @value];

   (* Set a given part to a new value *)
   If[! TrueQ[setPartDefined[s]],
     s /: setPart[s, pt_, value_] :=
       Module[{},
         savePartOnDisk[s, pt, value];
         releasePart[s, pt];
         value
       ];
     s /: setPartDefined[s] = True;
   ];
(* Release the API for this part. Irreversible *)
s /: releaseAPI[s, part] := Remove[sym];
];

What this does is to define a certain API for a symbol which represents the list in the framework. For lists of thousands parts, many thousands such UpValues are created. They are saved (serialized) when the list representation is saved for a later use, and read back in when this representation is loaded from disk. This is by far the most massive use of UpValues in my practice at least, and UpValues played a major role in my ability to structure the code this way, providing necessary means for encapsulation, instantiation and separation of interface and implementation. You can find more details in the linked discussion of the framework.

Another, somewhat similar, example is that of an implementation of mutable data structures in Mathematica. The way I do it is described here, and I will again use some code from that post to illustrate the point:

Module[{parent, children, value},
  children[_] := {};
  value[_] := Null;
  node /: new[node[]] := node[Unique[]];
  node /: node[tag_].getChildren[] := children[tag];
  node /: node[tag_].addChild[child_node, index_] := 
        children[tag] = Insert[children[tag], child, index];
  node /: node[tag_].removeChild[index_] := 
        children[tag] = Delete[children[tag], index];
  node /: node[tag_].getChild[index_] := children[tag][[index]];
  node /: node[tag_].getValue[] := value[tag];
  node /: node[tag_].setValue[val_] := value[tag] = val;
];

The node represents a new data type (tree node), and using UpValues allowed me to use the familiar dotted notation without hard overloading of Dot, and without giving the API symbols (addChild, getChild, etc) any meaning, so they can be reused by other data types.

There are many more uses for UpValues, but what I want to stress is that they are a very practical tool.

Answer 3

I am embarrassed to admit that I completely forgot about my own tutorial explaining how to use UpValues to create a lag operator, which I wrote sometime in the late 1990s and have had on my web site since then.

The UpValues construction is exactly what you need to define how certain built-in operators like arithmetic should behave in the context of a custom operator like a lag operator.

(Thanks to Murta for reminding me.)

Answer 4

An example I use from time to time is to "prettify" output. Suppose you have a not so huge matrix, e.g.

aa = Array[Subscript[a, #1, #2] &, {3, 3}]

which prints with commas between the Indexes. Sometimes, you don't want them and you can replace them with InvisibleComma. To do this, I use the following:

runocommaindex={Subscript[a_, b___, x_, y_, c___] -> 
  Subscript[a, b, Row[{x, "\[InvisibleComma]", y}], c]};
nokommaindex[expr_]:=(expr//.runocommaindex);

noKommaForm[expr_]:=KeineKommaForm[nokommaindex[expr]];
Format[KeineKommaForm[expr_]]:=expr;

noKommaMatrix[expr_]:=noKommaForm[MatrixForm[expr]];

rucommaback=Row[List[a_,"\[InvisibleComma]",b_]]->Sequence[a,b];
kommaback[expr_]:=(expr//.rucommaback);

KeineKommaForm/:Normal[KeineKommaForm[expr_]]:=kommaback[expr];
KeineKommaForm/:Normal[KeineKommaForm[MatrixForm[expr_]]]:=kommaback[expr];

noKommaMatrix displays the matrix in MatrixForm too. It's simply noKommaForm[MatrixForm[expr]]. Now you can switch off the display of the commas using Normal.

aa // noKommaForm
anm = aa // noKommaMatrix
Normal@anm

producing: