77
$\begingroup$

Recently I came across a set of problems which would be solved most easily within an object-oriented approach. I first attempted to solve them by other means, but found the complexity of the code growing too fast and, while individual functions used a mixture of styles, the overall feel and organization of the code was de-facto procedural. Having looked at the final code, I saw that I would have written it much faster and cleaner in Java, which is when I realized that OO was needed. Basically, I needed it to reuse some behavior and decrease coupling between components.

We all know that there were a number of attempts on implementing OO extensions for Mathematica. However, none of them IMO were very simple, and none of those I looked at had a natural "look and feel" for Mathematica (I could have missed some). OTOH, we also know that Mathematica has powerful metaprogramming capabilities, which should make it possible to implement the core of an OO extension rather easily.

Since OO is a broad paradigm, here are some requirements to restrict the problem. The desired OO extension should

  • Be idiomatic. Programming in it should feel natural for experienced Mathematica users. In particular, it should go well with immutable Mathematica expressions, and have the least possible problems concerning interoperability with Mathematica (garbage collection - related problems etc).

  • Support instantiation, inheritance and polymorphism. By inheritance I mean reuse of behavior (methods) rather than state (fields).

  • Reuse as much of the Mathematica's core constructs as possible

  • Limit the things that can be done with it as little as possible, compared to the programming techniques and tricks we are used to in Mathematica

  • If possible, have at least minimally convenient syntax for things like method calls.

  • Implementation should be simple in the sense that it should not introduce too many exterior constructs, such as ToString - ToExpression cycles, needless manipulations with contexts, creation of new symbols which are not strictly necessary, etc.

I am not so much worried about efficiency here, since the intended purpose for such an extension is more to help with larger-scale code organization for certain classes of tasks, than to be used with millions of light-weight objects.

In many previous questions on similar topics, the two rather separate topics were often mixed together. One concerns mutable struct-like data structures and their possible implementations in Mathematica, and the other was about OO proper. So, to clarify a bit more, I am not so much interested in the former topic here. I am rather more interested in the dynamic aspects (behavior reuse), and ways to conveniently package code and organize larger projects, but not so much in efficient mutable data structures. In other words, the subject matter is what can the object orientation bring to the table for larger Mathematica projects, and how can we make the most out of it in Mathematica.

So, the question is: can we have a core of such an OO extension in under 100 lines of Mathematica code?

$\endgroup$
9
  • $\begingroup$ @NasserM.Abbasi It is not free for one thing, and I really want a simple and short implementation for another one. $\endgroup$ Commented Dec 23, 2012 at 15:15
  • 2
    $\begingroup$ @NasserM.Abbasi This is not just on V9 - the form I used in undocumented. But it is the form I needed, and there isn't an easy replacement for it within the documented functionality. $\endgroup$ Commented Dec 23, 2012 at 15:34
  • $\begingroup$ Would you consider expanding this into a blog post eventually, after you've gotten some feedback? $\endgroup$
    – Szabolcs
    Commented Dec 25, 2012 at 1:08
  • 1
    $\begingroup$ Leonid, you are a modern marvel! :) Lately you seem to have been producing new and interesting projects at an incredible pace. I have little practical familiarity with the OO paradigm, but despite frequent calls for a Mathematica implementation I had tended to think it was probably not very useful in this context. That you hold the opposite view has surely made me think again, and I'm very much looking forward to having the time to give this the consideration it deserves! $\endgroup$ Commented Dec 25, 2012 at 3:38
  • $\begingroup$ @Szabolcs Yes, absolutely. That's the plan, actually. I know that it may seem like I've forgotten all my promises regarding the blog, but this isn't so. And for this topic, I posted this here since I wanted to attract more attention to this topic and make the core of this more accessible to anyone. I have much more stuff regarding OO, and that stuff will go into the blog post. For example, here I wasn't able to consider more meaningful / complex examples of applying this stuff to practical problems, but the blog will give me such an opportunity. $\endgroup$ Commented Dec 25, 2012 at 7:47

4 Answers 4

64
$\begingroup$

The answer seems to be yes. At least, I will try to describe an attempt which would pass the "under 100 lines of code" test. How well it satisfies the other criteria is a subjective matter, but I have already used it for larger-scale project with so far very positive results.

I will present two implementations. One is an absolute toy, but extremely simple. The other is also a toy, but having all the core requested features. The one I now use in real project is somewhat more complex, but has the same core.

A toy implementation in under 25 lines of code

Implementation

Here is the toy implementation. It is small enough that I can present it first and explain afterwards. The following implements the type declaration operator, where the type is represented by a symbol:

ClearAll[DeclareType];
DeclareType[s_Symbol] :=
  Function[Null,
    ClearAll[s];
    SetAttributes[s, HoldAll];
    defineMethods[s, ##];
    s,
    HoldAll];

This does not do much until defineMethods has been defined. Neither does this support inheritance, so we add the following:

DeclareType[s_Symbol ~ Extends ~ p_Symbol] :=
  Function[Null,
    DeclareType[s][##];
    SuperType[s] = p;
    s[self_][lhs_] := p[self][lhs];
    s,
    HoldAll];

Here, Extends is an inert head introduced for clarity only, and SuperType holds a value of the supertype, if there is any. Finally,

ClearAll[defineMethods];
SetAttributes[defineMethods, HoldRest];
defineMethods[s_] := s;
defineMethods[s_, SetDelayed[lhs_, rhs_], rest___] :=
  Module[{},
     s[cont_][lhs] :=              
        Block[{$self = s[cont], $super = SuperType[s][cont]}, 
          rhs
        ];
     defineMethods[s, rest]];

This is all there is here. This introduces two special symbols $self and $super, which can be used inside the body of any method defined for a type, to refer to the object itself or call methods of its supertype. The way this works is as follows: I use rules to define methods as SubValues for the type symbol, and the patterns used in the method definitions naturally become the parts of the resulting SubValues.

Examples

This declares the type Animal:

DeclareType[Animal][
   breathe[] := Print["I am breathing ", $self[[1, 1]]],
       sleep[duration_String: "one hour"] := 
           "I have been sleeping for " <> duration,
       sleep[duration_Integer] := 
          $self@sleep[ToString[duration] <> " hours"],
   move[] := Print["I move ", $self[[1,2]]]
]

(*  Animal *)

Now we create an instance of this type:

an = Animal[{"oxygen","fast"}]

(*  Animal[{oxygen,fast}]  *)

Note that the instance is completely stateless, it is fully defined by an expression (list) containing its content. We can now call methods:

an@breathe[]

During evaluation of In[68]:= I am breathing oxygen

an@sleep[]

(* I have been sleeping for one hour *)

an@sleep["two hours"]

(* "I have been sleeping for two hours" *)

an@sleep[5]

(*  "I have been sleeping for 5 hours"  *)

Already here, we can note one very important big advantage of this scheme: one can use the usual sweet Mathematica pattern-matching to define methods, including all (well, most of) the nice stuff, such as default values, overloading, etc. But what about inheritance? Well, here is an example:

DeclareType[Cat ~ Extends ~ Animal][
  sleep[] := StringJoin[$super@sleep[], $self[[1, 3]]]
]

(* Cat *)

We now create an instance:

cat = Cat[{"oxygen","fast"," on the floor"}]

(*  Cat[{oxygen,fast, on the floor}]  *)

and call the methods:

cat@breathe[]

During evaluation of In[125]:= I am breathing oxygen

cat@sleep[]

(* I have been sleeping for one hour on the floor  *)

cat@sleep["three hours"]

(* I have been sleeping for three hours *)

Note that inheritance works as one would expect. In particular, the Cat's version of sleep method is called for zero arguments, while Animal's version is called for other calls for sleep. Note also that the Cat's sleep method has access to the parent's method via $super.

In a sense, this is probably as close to an ideal mix of OO and Mathematica as it gets: objects are completely stateless and based on immutable Mathematica expressions. However, this approach has a number of flaws. In particular, it requires that all the symbols used as method names do not have definitions such that they can prematurely evaluate. For example, having defined something like

breathe[] := Print["breathe"] 

would spoil the above behavior. It is rather desirable that things be more robust. Another problem (which would persist to my other implementations) is shadowing. If two different users define types with the same method names which live in different contexts, shadowing will happen. It could probably be solved by converting symbol names to strings and then using only the short symbol names, but I think this is not an idiomatic Mathematica solution, in many ways. So, the other way to solve this (and this is what I do) is by convention: introduce a special context (e.g. OO`Methods`), and place all symbols used for method names there. This can also solve the evaluation problem, if in addition one adopts a convention to not assign any rules to those symbols. The above declaration for Animal would then look like:

DeclareType[Animal][
   OO`Methods`breathe[] := Print["I am breathing ", $self[[1, 1]]],
       OO`Methods`sleep[duration_String: "one hour"] := 
           "I have been sleeping for " <> duration,
       OO`Methods`sleep[duration_Integer] := 
          $self@OO`Methods`sleep[ToString[duration] <> " hours"],
   OO`Methods`move[] := Print["I move ", $self[[1,2]]]
]

Note that the user of the type does not have to use long names, as long as both the type's context and OO`Methods` are on the $ContextPath.

A more real thing

The implementation I am going to show now is the core of what I ended up using, but is based on the same set of ideas. The main technical difference is that, in order to prevent premature evaluation of method calls, I found no other way than to introduce some state into the objects / instances. They will be now represented by unique (within a given Mathematica session, but this restriction can be lifted) symbols. This is similar in spirit to how JLink implements references to Java objects. And I will use UpValues as a tool to attach both state and behavior to these symbols. While this scheme would help to solve the premature evaluation problem I mentioned, it also opens additional possibilities not easily possible in the simple setting of the previous section, such as attaching new methods to a single instance(rather than a type) at run-time. This by itself is a powerful enough feature to justify this approach (it is not supported by e.g. Java, but is supported by e.g. Javascript).

So, without further due, here is the new DeclareType:

ClearAll[DeclareType];
DeclareType[Object] = Null;
DeclareType[type_Symbol] := DeclareType[type~ Extends ~ Object];
DeclareType[Object ~ Extends ~ _] = Null;
DeclareType[type_Symbol ~ Extends ~ superType_Symbol] :=
   Function[
    Null
    ,
    ClearAll[type];         
    If[ValueQ[vtable[type]], vtable[type] =.];          
    SetAttributes[type, HoldAll];
    defineMethods[type, ##];
    SuperType[type] = superType;
    type
    ,
    HoldAll
];

It is not so much different from what we had before. The new symbols it relies on are defined as:

ClearAll[Object, object];
Object::nomethod = "Unknown method for type Object. The method call was `1`";
SetAttributes[{Object, object}, HoldAll];
Object[__] := object;
object[args___,methodCall_] := 
(
    Message[Object::nomethod,ToString@HoldForm[methodCall]];
    $Failed
);

(this is similar in spirit to Java Object - so that there is a single object hierarchy). Also, SuperType is now defined as

ClearAll[SuperType];
SuperType[Object] = Null;
SuperType[_] = Object;

A symbol vtable stores a reference to a dispatch function for any type. It is initially defined as

ClearAll[vtable];
vtable[Object]:=object;

but is dynamically acquires new definitions as new types are being defined. Here is a new defineMethods, and this is where things start to get substantially different:

ClearAll[defineMethods];
SetAttributes[defineMethods, HoldRest];
defineMethods[s_] := s;
defineMethods[s_, args___] :=
  Module[{ff},          
    SetAttributes[{ff}, HoldAll];
    vtable[s] = ff;
    s[content_] :=
       Module[{sym},                
          SetAttributes[sym, HoldAll];
          sym /: instanceValue[sym] = content;
          sym /: Normal[sym] := 
            With[{cont = instanceValue[sym]},
               HoldForm[s[cont]]
            ];
          (* Note: this forces the arguments to be evaluated *)
          sym[f_[argums___]] :=
            Hold[f][argums]/.Hold[h_][x___]:>
               ff[sym,f[x]];                    
          sym /: Set[sym, newcontent_] := 
            sym /: instanceValue[sym] = newcontent;                       
          sym/: TypeOf[sym] = s;                
          sym
       ];
    addMethods[args][s];
  ];

Here, the input symbol s stands for the name of the type. What happens here is that every time I call SomeType[some-content], I get a new symbol with the content attached via UpValues and the instanceValue container, and the behavior attached via DownValues and the ff symbol, which is a dispatch function for a given type. The latter is defined only once per type definition, and is stored in vtable[SomeType]. Also, the TypeOf operation is defined (similar to Java getClass), and one can also replace the full content of the object using Set. The instanceValue container is defined as

ClearAll[instanceValue];
SetAttributes[instanceValue,HoldAll];
instanceValue[_]:=
   Throw[$Failed, {instanceValue,"invalid_instance"}];

The final piece we need is addMethods. Here is the code:

ClearAll[addMethods];
SetAttributes[addMethods, HoldAll]; 
addMethods[SetDelayed[lhs_, rhs_], rest___][s_] :=
  With[{ff = vtable[s]},
    ff[sym_, lhs] := 
       Block[{
         $self = sym, 
             $content = instanceValue[sym],
         $super    				
             }
             ,
             SetAttributes[$super,HoldAll];
         $content /: Set[$content, expr_] := Set[sym, expr];
         $super[call_]:= vtable[SuperType[s]][sym,call];
         rhs        
       ];
       addMethods[rest][s]
  ];

addMethods[][s_] :=
   vtable[s][sym_, lhs_] := vtable[SuperType[s]][sym,lhs];  

What happens here is that the definitions for the type's dispatch function ff are formed based on the definitions of the methods, supplied by the user. Again, we win big due to the generality of patterns, which allows us to automatically create valid definitions for ff by doing a scope surgery on the original definitions of the methods. Another thing worth noting is again the user of Block which allows the user to use symbols $self, $super and $content inside a body of a method.

The same examples as before should work here. In one piece, the code lives here. I left out some more advanced things such as formatting, per-instance method addition at run-time, automatic object release (when objects are no longer needed), etc. These are all implemented in the complete full version that lives here. In particular, the full version has a very simple installation procedure described in the README file in that linked gist. It has an example notebook, which is opened by a simple command GetExampleNotebook[], once the project has been installed and loaded.

Use cases and limitations

The OO extension described above is intended to be used as a tool for large-scale project organization. In this capacity, I tested it already on real projects and so far have had very positive results, in terms of code complexity management, modularization of component, and making code more reusable.

Some real use cases

Here I will list some real examples which rely on it:

I will add more documentation to these, so that it will be easier to explore them, but in fact they are very easy to use.

The code of these example projects also shows how one can do practical things, such as attaching or changing the state of an object. The basic idea is that it is better to avoid adding explicit state (in fact, the current version of OO does not support object fields, only object methods. This was intentional, but such a support can be easily added if there will be a consensus that they are needed). Instead, one can add new methods which would encapsulate the state. This allows to avoid introducing mutable state in objects in many cases, which arguably leads to a cleaner code.

Limitations

This OO extension has not been designed with high-performance applications in mind. While you can create thousands of objects and use it also for lightweight objects, you will face more or less the usual high-level Mathematica overhead (in my view, this is not so much of a real limitation, since Mathematica has other means for high-performance programming, and OO is often not really needed on such a "fine-grained" scale). Also, one must be careful to avoid the memory leaks.

Certain types of patterns in method definitions may also not work, in particular conditional patterns where condition is attached to the left-hand side, such as

myMethod[x_,y_]/;x<y:=body

Again, this limitation can be rather easily removed by making a slightly more complex parser for addMethods. OTOH, the definitions like

myMethod[x_,y_]:=body/;x<y

should work, so this is perhaps not so much of an issue.

Further developments

While I feel that the current state of OO is minimally complete, in the sense that it can be used in serious applications, I plan some further developments, such as

  • Reflection a-la Java (meta-information on methods, etc)
  • Integrated unit-tests
  • Some other enhancements

The extension was initially build not just for its own sake, but to provide a base for the development of other projects of interest to me (and very possibly also to a wider audience). So I expect that it will be further shaped by the needs of those projects people build using it. So, any feedback on the system is much appreciated, and will affect further developments.

$\endgroup$
6
  • $\begingroup$ Strap in the for ride: here we go again! :-D $\endgroup$
    – Mr.Wizard
    Commented Dec 23, 2012 at 15:30
  • 1
    $\begingroup$ @Mr.Wizard I have to go now, but will do it later today. $\endgroup$ Commented Dec 23, 2012 at 15:47
  • 2
    $\begingroup$ Could sym[f_[argums___]] := Hold[f][argums]/.Hold[h_][x___]:> ff[sym,f[x]]; be replaced with sym[f_[argums___]] := {argums} /. {x___} :> ff[sym,f[x]];? $\endgroup$
    – Mr.Wizard
    Commented Dec 23, 2012 at 15:51
  • $\begingroup$ @Mr.Wizard Probably yes. I guess the first one was a remnant from some original code, and I did not rethink it after changes. Thanks, I will edit this in soon. $\endgroup$ Commented Dec 23, 2012 at 15:54
  • $\begingroup$ I suppose sym[f_[argums___]] := ff[sym,f[##]] &[argums]; might work too. $\endgroup$
    – Mr.Wizard
    Commented Dec 23, 2012 at 17:20
31
$\begingroup$

I know this thread is old and that Leonid provided a great solution, but I'd like to present a bit different approach.

My approach is inspired by JavaScript. There are no classes, just "pure objects".

To directly answer requirements from question

  • Support instantiation, inheritance and polymorphism.

    Objects are fully dynamic, everything can be changed in run-time. Objects can inherit behavior (and state if that's needed) directly from other objects. There are no classes (types), some form of polymorphism can be achieved through duck typing.

  • Reuse as much of the Mathematica's core constructs as possible

    Objects are implemented as symbols with state and behavior encoded purely in ordinary DownValues.

  • If possible, have at least minimally convenient syntax

    Attributes and methods are accessed like this: obj@attr, obj@method[args] and set using ordinary Set and SetDelayed functions.

  • Implementation should be simple

    There are no ToString - ToExpression cycles, no manipulations with contexts, no auxiliary symbols are created, everything is encoded in symbols explicitly declared as objects by the user.

  • "Be idiomatic" and "Limit the things that can be done with it as little as possible" points are subjective, I hope those conditions are fulfilled.

Leonid started his answer with toy implementation that showed core concepts in simplest form possible, it helped me a lot in understanding them, so I'll also start with a toy model.

Toy implementation

ClearAll[DeclareObject];
DeclareObject[obj_Symbol] := (
    ClearAll[obj];
    obj /: Set[obj[lhs_], rhs_] := (setMember[obj, lhs, rhs]);
    obj /: SetDelayed[obj[lhs_], rhs_] := (bindMember[obj, lhs, rhs]);
)

Above function declares symbol as object. obj /: ... lines is a syntactic sugar that allows object members to be defined using ordinary Set and SetDelayed syntax, those members will be automatically made inheritable by setMember and bindMember functions.

ClearAll[WithOrdinaryObjectSet];
SetAttributes[WithOrdinaryObjectSet, HoldRest];
WithOrdinaryObjectSet[obj_Symbol, body_] :=
    With[
        {
            oldUpValues = UpValues[obj],
            protected = Unprotect[obj]
        }
        ,
        UpValues[obj] = 
            FilterRules[
                UpValues[obj],
                Except[HoldPattern[HoldPattern][_Set | _SetDelayed]]
            ];
        Protect[protected];

        body;

        Unprotect[protected];
        UpValues[obj] = oldUpValues;
        Protect[protected];
    ]

WithOrdinaryObjectSet function temporarily switches off "set-altering" up values from object and lets us define ordinary down values without using setMember and bindMember functions.

ClearAll[setMember, bindMember];
SetAttributes[{setMember, bindMember}, HoldRest];
setMember[obj_Symbol, lhs_, rhs_] :=
    WithOrdinaryObjectSet[obj,
        obj[lhs] = rhs;
        obj[lhs, _] = rhs
    ]
bindMember[obj_Symbol, lhs_, rhs_] :=
    WithOrdinaryObjectSet[obj,
        obj[lhs] := Block[{$self = obj}, rhs];
        obj[lhs, self_] := Block[{$self = self}, rhs]
    ]

setMember takes an object and left- and right hand sides of member definition. It temporarily switches off "set-altering" up values from object to avoid recursion. Then sets two down value definitions, on given object. First is for member called directly on given object and second is used when member is called on objects inheriting from given object.

bindMember, in addition to setting two down values like setMember, provides $self symbol that can be used in rhs of "bound" member definition. When bound member is called on any object value of $self becomes this object.

ClearAll[SetSuper, Super];
SetSuper[obj_Symbol, super_Symbol] := (
    Super[obj] ^= super;
    WithOrdinaryObjectSet[obj,
        obj[x_] := super[x, obj];
        obj[x_, self_] := super[x, self]
    ]
)

SetSuper is a function that makes one object obj inherit from another object super.

The line obj[x_] := super[x, obj] is the core of our inheritance, if member is not defined on obj it will delegate member call to it's parent passing also itself as second argument (this is used to correctly set $self variable for bound member call).

Line obj[x_, self_] := super[x, self] allows passing of member calls delegated by descendants of obj.

Super is a function that returns parent of given object.

That's the core of the system. For convenience we'll add one more "shorthand" definition to quickly define inheriting objects.

DeclareObject[obj_Symbol, super_Symbol] := (
    DeclareObject[obj];
    SetSuper[obj, super]
)

Examples

In our approach there are no classes, but one can define a "prototype object" that performs similar role. So let's start with animal prototype (something that has similar role to Animal type from Leonids answer).

DeclareObject[animalPrototype];
animalPrototype@breathe[] := Print["I am breathing ", $self@breathingGas]
animalPrototype@sleep[duration_String: "one hour"] := "I have been sleeping for " <> duration
animalPrototype@sleep[duration_Integer] := $self@sleep[ToString[duration] <> " hours"]
animalPrototype@move[] := Print["I move ", $self@speed]

Note that all members are set "in run-time" on already existing object. We used members breathingGas and speed instead of parts of expression since our objects are symbols and state is encoded in down values not in object expression.

When inspecting animalPrototype object we see pairs of bound members

??animalPrototype
(*
Global`animalPrototype
(animalPrototype[lhs$_]=rhs$_)^:=setMember[animalPrototype,lhs$,rhs$]
(animalPrototype[lhs$_]:=rhs$_)^:=bindMember[animalPrototype,lhs$,rhs$]
animalPrototype[breathe[]]:=Block[{$self=animalPrototype},Print[I am breathing ,$self[breathingGas]]]
animalPrototype[move[]]:=Block[{$self=animalPrototype},Print[I move ,$self[speed]]]
animalPrototype[breathe[],self$_]:=Block[{$self=self$},Print[I am breathing ,$self[breathingGas]]]
animalPrototype[sleep[duration_String:one hour]]:=Block[{$self=animalPrototype},I have been sleeping for <>duration]
animalPrototype[sleep[duration_String:one hour],self$_]:=Block[{$self=self$},I have been sleeping for <>duration]
animalPrototype[sleep[duration_Integer]]:=Block[{$self=animalPrototype},$self[sleep[ToString[duration]<> hours]]]
animalPrototype[sleep[duration_Integer],self$_]:=Block[{$self=self$},$self[sleep[ToString[duration]<> hours]]]
animalPrototype[move[],self$_]:=Block[{$self=self$},Print[I move ,$self[speed]]]
*)

Now let's define object inheriting from prototype (something analogous to instance of Animal type) and let's set needed members. We could have also set default values of those members on prototype object (something similar to e.g. class variables in Python).

DeclareObject[an, animalPrototype];
an@breathingGas = "nitrogen";
an@speed = "slow";

Inspecting an object we can see pairs of unbound members breathingGas and speed and definitions an[x$_]:=animalPrototype[x$,an], an[x$_,self$_]:=animalPrototype[x$,self$] that make an inherit from animalPrototype

??an
(*
Global`an
Super[an]^:=animalPrototype
(an[lhs$_]=rhs$_)^:=setMember[an,lhs$,rhs$]
(an[lhs$_]:=rhs$_)^:=bindMember[an,lhs$,rhs$]
an[breathingGas]=nitrogen
an[speed]=slow
an[x$_]:=animalPrototype[x$,an]
an[breathingGas,_]=nitrogen
an[speed,_]=slow
an[x$_,self$_]:=animalPrototype[x$,self$]
*)

Let's check that methods work as expected

an@breathe[]
(* During evaluation of In[18]:= I am breathing nitrogen *)
an@sleep[]
(* "I have been sleeping for one hour" *)
an@sleep["two hours"]
(* "I have been sleeping for two hours" *)
an@sleep[5]
(* "I have been sleeping for 5 hours" *)

Now we could define catPrototype object to mimic Cat class, but to show differences between this and Leonids version let's define cat object directly inheriting from animalPrototype and set overridden method directly on this object.

DeclareObject[cat, animalPrototype]
cat@breathingGas = "oxygen";
cat@speed = "fast";
cat@lair = " on the floor";
cat@sleep[] := StringJoin[animalPrototype[sleep[], $self], $self@lair]

We used animalPrototype[sleep[], $self] to call non-overridden sleep[] member from animalPrototype object.

Instead of animalPrototype[sleep[], $self], we could have used Super[cat][sleep[], $self] to always use member from current parent of cat. Such use case is similar to python built-in super function.

obj[member, otherObj] is a very powerful construct, we can call member from any object passing as $self any other object, those objects don't have to be related via inheritance.

Overridden and non-overridden methods work as expected

cat@breathe[]
(* During evaluation of In[32]:= I am breathing oxygen *)
cat@sleep[]
(* "I have been sleeping for one hour on the floor" *)
cat@sleep["three hours"]
(* "I have been sleeping for three hours" *)

Object can inherit from any other object so let's define cat2 object inheriting from cat

DeclareObject[cat2, cat]
cat2@lair = " on the couch";

Since we've overridden only lair member, cat2 inherits from cat not only bound members, but also unbound members breathingGas and speed.

cat2@breathingGas
(* "oxygen" *)
cat2@speed
(* "fast" *)
cat2@lair
(* " on the couch" *)
cat2@breathe[]
(* During evaluation of In[48]:= I am breathing oxygen *)
cat2@sleep[]
(* "I have been sleeping for one hour on the couch" *)
cat2@sleep["three hours"]
(* "I have been sleeping for three hours" *)

If we change state of cat non-overridden state of cat2 will also change.

cat@speed = "very fast";
cat2@speed
(* "very fast" *)

Overridden members of cat2 are not affected by changes in cat

cat@lair = " on the chair";
cat2@lair
(* " on the couch" *)

If we unset member from cat2 its parent member will be used

cat2@lair =.
cat2@lair
(* " on the chair" *)

"Constructors"

We can define a "constructor" for objects inheriting from a prototype:

animalPrototype@construct[obj_Symbol, breathingGas$_, speed$_] := (
    DeclareObject[obj, $self]
    obj@breathingGas = breathingGas$;
    obj@speed = speed$;
    obj
)

Note that there's nothing special/reserved/built-in in the constructor, it's an ordinary user defined member.

Let's create an object using our constructor.

animalPrototype@construct[spaceWhale, "nothing", "at warp 10"];
spaceWhale@breathe[]
(* During evaluation of In[58]:= I am breathing nothing *)
spaceWhale@move[]
(* "I move at warp 10" *)

Protected objects

If we don't want to allow further changes of an object, e.g. we want to make sure a prototype will not change anymore in run time, we can simply Protect it as any other symbol.

Protect[animalPrototype];

We can no longer change members of protected object.

animalPrototype@move[] =.
(*
During evaluation of In[61]:= Unset::write: Tag animalPrototype in animalPrototype[move[]] is Protected. >>
$Failed
*)
animalPrototype@breathe[] = "completly new concept of breathing"
(*
During evaluation of In[73]:= Set::write: Tag animalPrototype in animalPrototype[breathe[]] is Protected. >>
During evaluation of In[73]:= Set::write: Tag animalPrototype in animalPrototype[breathe[],_] is Protected. >>
*)

A bit chaotic summary of basic concepts

  • An object is a symbol with some special up values.

    DeclareObject[symbol] function is intended for setting those up values on symbols.

  • Object obj inherits from other object otherObj if it has special down value definitions:

    • obj[x_] := otherObj[x, obj].

      If obj@something will not match any other pattern this down value will delegate call to otherObj.

    • obj[x_, self_] := otherObj[x, self]

      This definition transfers calls from child of obj to otherObj.

    DeclareObject[obj, otherObj] is intended for declaring symbol as inheriting object. SetSuper[obj, otherObj] is intended for changing parent of already existing symbol.

  • Member of an object obj is any pattern member, except _, for which obj has defined down value obj[member] or obj[member, self_]. Based on existence of above down values we can distinguish three kinds of members.

    • Non-inheritable member has only obj[member] definition so it can be called directly on obj like obj@member, but it will be ignored by calls delegated to obj by objects inheriting from obj.
    • Inheritable-only member has only obj[member, self_] definition. It will be matched by calls delegated from objects inheriting from obj, but will not be matched when called directly on obj: obj@member.
    • Inheritable member has both definitions so will be matched when called directly on obj and when it's call is delegated from objects inheriting from obj.

    Above distinction was based on "left hand side" of member definitions, there's another useful distinction based on "right hand side" of definition.

    • Bound member has definition with RHS of form: Block[{$self = ...}, ...]. Inside body of "right hand side" of bound member definition, $self symbol represents object on which member was called.
    • Unbound member does not attach any special meaning to $self symbol.

Some of "special up values" set on symbols by DeclareObject function override Set and SetDelayed functions.

  • Using Set will, by default, define inheritable unbound member so obj@member = value will additionally set obj[member, _] = value definition.

  • SetDelayed will, by default, define inheritable bound member. obj@member := body will define obj[member] := Block[{$self = obj}, body] and obj[member, self_] := Block[{$self = self}, body]. So inside body, we can use $self symbol and it will denote object on which member was called.

Non-inheritable, inheritable-only and unbound delayed members must be set inside body of WithOrdinaryObjectSet function that temporarily switches off "set altering up values" from given object.

In this approach difference between "state" and "behavior" is just a convention. "Behavior" (methods) are implemented in the same way as state (attributes) they're both ordinary down values. Only difference is that down value patterns defined with SetDelayed are bound to objects and patterns defined with Set are not bound.

Leonids concerns of shadowing and premature method evaluation are also valid in case of my approach. The difference is that in this case since methods are implemented as down values, not sub values, one could add HoldFirst attribute to symbol representing an object (or to DeclareObject function to add it by default to all objects).

Non-toy implementation

Real implementation is in ClasslessObjects package, that is available on GitHub.

Package is very similar to toy implementation presented above. Differences include:

  • Object symbol that is an ancestor of all objects. If member delegation reaches Object warning is printed and $Failed is returned. An idea taken directly from Leonids package.
  • unsetMember function and up value that overrides Unset on objects.
  • SetSuper is protected from cyclic inheritance.
  • ObjectQ function recognizing objects.
$\endgroup$
7
  • $\begingroup$ Very interesting, +1. I always wanted to construct a prototype-based version too (particularly because, while I do use my own version a lot, I tend to use it in a prototype-style fashion, adding behavior to particular instances). Have no time for more than a quick look today, but will look closer tomorrow. $\endgroup$ Commented Nov 8, 2014 at 0:48
  • $\begingroup$ +1 Nice work, I'm interested in your full version. Object oriented programming is sometimes useful even in Mathematica and your syntax is really easy, I like it. $\endgroup$
    – faysou
    Commented Nov 8, 2014 at 1:22
  • $\begingroup$ +1 i'm definitely interested in the non toy version! $\endgroup$
    – P. Fonseca
    Commented Nov 8, 2014 at 10:55
  • $\begingroup$ @Leonid Previous toy implementation was broken. It passed $self only one level in inheritance hierarchy. I've edited my answer fixing toy implementation. $\endgroup$
    – jkuczm
    Commented Nov 10, 2014 at 8:52
  • $\begingroup$ @faysou I've added a brief comparison of toy and full version. $\endgroup$
    – jkuczm
    Commented Nov 10, 2014 at 8:56
23
$\begingroup$

After years of development, I'm releasing a package called MTools on github.
The package is under an MIT license. You can fork it and send pull requests.

The main contribution of MTools is to allow object oriented programming in Mathematica in a very natural way.

The package also contains:

  • Generic classes for manipulating trees of objects and displaying them
  • Automatic interface generation for displaying and editing objects
  • Functions for doing asynchronous evaluation easily using parallel kernels (MSync)
  • Tools for accessing Couchbase, serializing and deserializing objects.

If you're curious about the code, the most interesting parts are here.

Another post on Wolfram Community talks about the package as well.


Installation

The package is located here https://github.com/faysou/MTools

Concepts

The package is based on ideas I exposed in my answer to Struct equivalent in Mathematica?. At the time Association didn't exist yet.

The idea is that an object is a Class[object] element where object is a symbol that stores an Association and Class has a HoldFirst attribute. This allows to store functions in Class as UpValues and properties in the object Association.

Functions are usually written using this form

Class.f[x_] := x; 

And stored using this form

Class /: o_Class.this.f[x_] := x;

The conversion happens with an UpValue rule in Dot.

To define a class you usually use the NewClass function, which takes a "Fields" and a "Parents" option as argument.

TestClass = NewClass["Fields"->{"a","b"->1},"Parents"->{Parent2, Parent1}]
Options@TestClass

All classes inherit the BaseClass class, that implements common setters and getters.
GenericClass and GenericGroup contain a lot of useful common functions for managing and displaying trees of objects.

"Fields" allows to define variables with default arguments. Here "a" will have a None default value. Default values are stored in the Options of the class.
"Parents" defines the super classes of a class.

If you already have objects created and you wish to add functions to a class, you can, but you then need to call ResetClasses[] for the new function to be recognised.

Method resolution

Let's give an exemple to understand how inheritance works and how function resolution is done.

(*class definitions*)
X=NewClass["Fields"->{"a"->1}]
X.f[]:=o["a"] (*o is the object representation in a method*)
X.g[x_]:=22

Y=NewClass["Parents"->{X}]
Y.f[]:=3
Y.g[x_?EvenQ]:= o.f[]

Z=NewClass["Parents"->{Y}]
Z.f[]:=4

(*Z object definition*)
zz=New[Z]["a"->2]

(*setters*)
zz.set["b",2]
zz.a = 3

(*getters*)
zz["a"]
zz.a

(*function calls*)
zz.f[] (*defined in Z*)
zz.super.f[] (*defined in Y*)
zz.super[X].f[] (*defined in X*)

zz.g[2] (*defined in Y*)
zz.g[3] (*defined in X*)

When NewClass is called a list of parents is created so that a class can find in which class a method is implemented.

In this exemple we will have

 Supers[Z] == {X, Y}

When a function from a Z object is executed it will search definitions first in Z, then Y, then X. The super keyword allows to execute definition from a super class.
So zz.f[] will execute the definition stored in class Z and using super, you can access implementations of f from other super classes.

The method resolution is similar to what is done in Python, in fact I realized I had done the same thing quite recently.

Once a method resolution has been done, this search is cached. For a normal function call like zz.f[] it will be cached in a symbol called sub, while for a super class call it will be cached in the super symbol.

Functions can be overloaded for certain patterns only and this is very powerful. For exemple for the zz.g[3] call the function will first try to use the definition from Y, but as it will return unevaluated a super call from the point of view of Y will be automatically done in order to reach the defintion of X.

Internally the following rewrites will happen:

Z[object$1].g[3] -> 
defineSub[Z, {object$1}, g, {3}] (*called only once then cached*) ->
Y[object$1,Z].this.g[3] -> 
Y[object$1,Z].super.g[3] -> 
X[object$1,Y,Z].this.g[3]

Notice a "class stack" after object$1 used in the method resolution when executing definitions in different classes.

Edit 15/2/2023: After many years asking myself how I could avoid the rule that was responsible for method resolution (HoldPattern[class[params___].(f:Except[super])[args___] /; !TrueQ[$blockSub[class, f]] ] :> class[params].sub.f[args]), I managed to find another mechanism that can rely solely on the traditional mathematica evaluation using UpValues.

The key idea consists in storing method definitions using the form o_MyClass.this.f[x_] instead of o_MyClass.f[x_] (the rewriting is done automatically, existing code still works). This then avoids having to intercept method calls of the form o.f[] using the rule above that was required previously.

There was also a sub funtion responsible for finding the definition of which class should be used (Class[object].sub.f[]) that is not used anymore, that's why I'm bumping the version by 1 in the first part of the version.

My estimate is that the new method resolution is at least two times faster than before.

$\endgroup$
11
  • $\begingroup$ You could put your package on PackageData.net $\endgroup$
    – Szabolcs
    Commented Jul 3, 2016 at 20:13
  • $\begingroup$ OK I'm going to. $\endgroup$
    – faysou
    Commented Jul 3, 2016 at 20:21
  • 2
    $\begingroup$ Nice work, man! $\endgroup$ Commented Jul 4, 2016 at 8:01
  • 2
    $\begingroup$ Than you Leonid. I remember when I started programming in Mathematica and was reading some of your posts, I didn't understand all this Hold stuff. This lead me to being interested in non standard evaluation. I'm happy to have done something useful from it and share it. I'm quite curious about if people wil use this package or not. It's a few thousands of hours worth of work. In any case it's useful for me. $\endgroup$
    – faysou
    Commented Jul 13, 2016 at 19:41
  • $\begingroup$ @faysou: Thanks for writing MTools. Is there a way to add properties which are computed on initialization of the object: X.b = o["a"]^2? One could create a function for it: X.b[] = o["a"]^2 but in this case the result of this calc is not saved anywhere, and will be repeated each time X.b[] is called. Of course, one can cache it manually on the instance level: x.b = X.b[], but that means writing it for every object. $\endgroup$
    – Adobe
    Commented Sep 8, 2018 at 16:42
10
$\begingroup$

So I think the language has matured enough to make good OOP possible in pure WL, if not entirely obvious. There are two things that allow it to work well, 1) Association and the fact that it supports a vectorized form for Lookup 2) Various low-level tricks like SetMutationHandler and SetNoEntry.

My basic idea was to imitate python in my setup, but do it in a vectorized manner. I store all my data in a single symbol, and create an OFObject (the OF for ObjectFramework) and OFType which implement a class and object interface (although it could easily be changed to support classless objects which would make things a little bit faster).

I used SetNoEntry so I could naturally support a part syntax allowing for in-place property modification, a la Association with SetMutationHandler. That plus vectorized lookups and sets overcomes some of the inefficiency of OOP implemented at top-level.

Unfortunately to detail everything here would be to run way over SE's character limit but I detailed it here.

My package is here and if people are interested I can write up some docs for it. This was largely an academic exercise for me, as I implemented a different, very similar OOP framework for the chemistry stuff that I actually use, but maybe it'll be useful for others.

Here's a quick example of what the kind of syntax it uses. Here's defining a type:

ballInABox:=OFBegin["BallInABox"];
OFInit[self_,
    position:{_?NumericQ,_?NumericQ}:{0,0},
    velocity:{_?NumericQ,_?NumericQ}:{1,1}
    ]:=
    OFSetThread[self,{"Position","Velocity"},{position,velocity}];
OFField["Box"]={{-5,5},{-5,5}};
OFField["ElasticityMultiplier"]=1;
OFMethod["Step"][
  self_,
  time : _Quantity : Quantity[.1, "Seconds"],
  steps : _Integer?Positive : 1
  ] :=
 Module[{position, velocity, box, mulp, 
   timenum = QuantityMagnitude@UnitConvert[time, "Seconds"]},   
  {position, velocity, box, mulp} =
   OFLookup[
    self, {"Position", "Velocity", "Box", "ElasticityMultiplier"}];
  (self[{"Postion", "Velocity"}] = {position, velocity}; #) &@
   Reap[
     Do[
      position += velocity*timenum;
      Which[
       position[[1]] < box[[1, 1]],
       position[[1]] = box[[1, 1]];
       velocity[[1]] = -mulp*velocity[[1]],
       position[[1]] > box[[1, 2]],
       position[[1]] = box[[1, 2]];
       velocity[[1]] = -mulp*velocity[[1]]
       ];
      Which[
       position[[2]] < box[[2, 1]],
       position[[2]] = box[[2, 1]];
       velocity[[2]] = -mulp*velocity[[2]],
       position[[2]] > box[[2, 2]],
       position[[2]] = box[[2, 2]];
       velocity[[2]] = -mulp*velocity[[2]]
       ];
      Sow@position,
      steps
      ]
     ][[2, 1]]
  ]
OFEnd[]

ball-in-a-box

And here's using it:

ball=ballInABox[RandomReal[{-5,5},2],RandomReal[{-2.5,2.5},2]]

ball-object

With[{box=ball["Box"]},
    ListAnimate[
        Graphics[Point[#],PlotRange->box]&/@ball["Step"][100]
        ]
    ]

bounce-gif

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.