Short syntax for accessing System`Utilities`HashTableAdd and System`Utilities`HashTableGet

Question

I have a need for a struct analogue in Mathematica. After reading this post I came to the conclusion that System`Utilities`HashTable is one of the best options. But it's rather terse to write. Let's say we have a hash table:

h=System`Utilities`HashTable[];

Is there a way to auto-magically transform

h.somekey=value (*would be evaluated as===> System`Utilities`HashTableAdd[h, "somekey", value]*)

and

a = h.somekey (*would be evaluated as ===> a = System`Utilities`HashTableGet[h, "somekey"]*)

I have not yet been able to find the right combination of := or :> to get this to work, but RuleDelayed is still a bit mysterious for me:)

---

EDIT

Some test cases:

h = System`Utilities`HashTable[];
h.first = "test";
h.second = "another test";
System`Utilities`HashTableAdd[h, "third", "yet another..."];
h.first    (*test*)
h.second  (*another test*)
h.third   (*yet another...*)
System`Utilities`HashTableGet[h, "first"]
System`Utilities`HashTableGet[h, "second"]
System`Utilities`HashTableGet[h, "third"]
(* is it possible to have dot hold the second argument only in the \
overloaded definition? so that this would still work?*)
second = a;
h.second (*should return: but does not another test*)
h.Unevaluated[second]
h."second"
Clear[second]
h.first = FIRST;
h.first
(*don't break the dot prodocut*)
Clear[a, b, c, d];
{a, b}.{c, d}
c = 5;
{a, b}.{c, d}

The obvious does not work:

Unprotect[Dot];
Dot[h_System`Utilities`HashTable, key_] := 
  System`Utilities`HashTableGet[h, ToString[Unevaluated[key]]];
Dot /: Set[Dot[h_System`Utilities`HashTable, key_], value_] := 
  System`Utilities`HashTableAdd[h, ToString[Unevaluated[key]], value];

If one leaves just h_ then it works, but breaks the dot product definition. As a work around this sort of works...

Unprotect[Dot, Set];
Dot[h_, key_Symbol] := 
  System`Utilities`HashTableGet[h, ToString[Unevaluated[key]]];
Dot /: Set[Dot[h_, key_Symbol], value_] := 
  System`Utilities`HashTableAdd[h, ToString[Unevaluated[key]], value];

So, why doesn't the first version work?

Also, is it possible to get Dot to hold its argument only for the overloaded version?

Answer 1

If you're willing to use a slightly different syntax to invoke System`Utilities`HashTableAdd, you can create your own wrapper around System`Utilities`HashTable that does most of what you want without modifying any built-in functions. The loss of the convenient hashTable.key = value syntax is unfortunately necessary because you can't use TagSet to set a tag more than one level deep in an expression, but personally I think that the postfix form hashTable.key.set[value] isn't much worse. Edit: by introducing a wrapper symbol specifically for this functionality it is also possible to overload Dot reasonably safely and enable the use of infix form as per the question (see below).

Here's the code:

ClearAll[hashTableWrapper];

(* Create new hash table *)
hashTableWrapper /: new[hashTableWrapper] :=
  hashTableWrapper@System`Utilities`HashTable[];

(* Check existence of a key *)
hashTableWrapper /: Dot[
   hashTableWrapper[h_System`Utilities`HashTable], key_, existsQ[]
  ] := System`Utilities`HashTableContainsQ[h, Unevaluated[key]];

(* Clear key value/delete key *)
hashTableWrapper /: Dot[
   hashTableWrapper[h_System`Utilities`HashTable], key_, clear[]
  ] /; System`Utilities`HashTableContainsQ[h, Unevaluated[key]] :=
  System`Utilities`HashTableRemove[h, Unevaluated[key]];

(* Nonexistent keys can't be cleared, but let's not complain *)
hashTableWrapper /: Dot[
   hashTableWrapper[h_System`Utilities`HashTable], key_, clear[]
  ] = Null;

(* Set key value (clearing first if necessary) *)
ClearAll[set]; SetAttributes[set, SequenceHold];
hashTableWrapper /: Dot[
   w : hashTableWrapper[h_System`Utilities`HashTable], key_, set[val___]
  ] := (
   w.Unevaluated[key].clear[];
   System`Utilities`HashTableAdd[h, Unevaluated[key], Unevaluated[val]];
   val
 );

(* Set delayed key value (again, clearing first if necessary) *)
ClearAll[setDelayed]; SetAttributes[setDelayed, {HoldAll, SequenceHold}];
hashTableWrapper /: Dot[
   w : hashTableWrapper[h_System`Utilities`HashTable], key_, setDelayed[val___]
  ] := (
   w.Unevaluated[key].clear[];
   System`Utilities`HashTableAdd[h, Unevaluated[key], Unevaluated[val]]
 );

(* Get key value *)
hashTableWrapper /: Dot[
   hashTableWrapper[h_System`Utilities`HashTable], key_, optional : get[] : get[]
  ] /; System`Utilities`HashTableContainsQ[h, Unevaluated[key]] :=
  System`Utilities`HashTableGet[h, Unevaluated[key]];

(* Deal with query of nonexistent key *)
hashTableWrapper /: Dot[
   hashTableWrapper[h_System`Utilities`HashTable], key_, optional : get[] : get[]
  ] = $Failed;

As a demonstration, let's say we find ourselves forgetting what 1 + 1 is and want to store it in a hash table. And perhaps we want to store the inverse of this operation as well. (These contrived examples help to demonstrate usage with unevaluated expressions for both the key and the value.)

hash = new[hashTableWrapper];
hash.Unevaluated[1 + 1].set[2];
hash.(2).setDelayed[Print["the value was evaluated"]; 1 + 1];

As this example shows, the key is evaluated unless otherwise specified using Unevaluated. (The parentheses in hash.(2).set[...] are necessary as otherwise this will be interpreted as a multiplication: 0.2 hash set[...].) Now we can write:

hash.Unevaluated[1 + 1].get[]

or simply (since get[] is an optional argument)

hash.Unevaluated[1 + 1]

and get back 2. Similarly, if we write hash.(2).get[] or hash.(2), "the value was evaluated" is printed and the unevaluated value 1 + 1 is evaluated to return 2.

Maybe we think that the second definition isn't very useful since it'll be evaluated every time. So, we redefine it:

hash.(2).set@Unevaluated[1 + 1];

which clears the previous definition automatically. (Printing "the value was evaluated" in the process. This is the result of System`Utilities`HashTableRemove's own behaviour which is to return the value associated with the key that was removed, if any.) Now hash.(2) will give, more usefully, Unevaluated[1 + 1].

Finally we come to our senses and decide to get rid of these trivial hash table entries. Again, because this calls System`Utilities`HashTableRemove, the values of the keys to be cleared are returned:

hash.Unevaluated[1 + 1].clear[]

2

and

hash.(2).clear[]

Unevaluated[1 + 1]

System`Utilities`HashTableRemove throws an error if invoked on a nonexistent key, but clearing such in this way simply does nothing, returning Null, as I thought this behaviour was more useful. However, an attempt to read undefined keys still fails:

hash.undefined

$Failed

Incidentally, you can have keys containing Dot without any problems, thanks to Dot's attributes Flat and OneIdentity. This one contains Dot in two different ways:

hash.this.is.a.key.containing.Dot.set["hello"];
hash.this.is.a.key.containing.Dot

"hello"

Edit: implementing infix syntax for Set

Leonid makes a point very well here regarding overloading Set, which actually applies to any built-in function. When overloading any built-in, one has to ask oneself:

If two people did this at the same time, without being aware of each other, could it result in a conflict?

Obviously, the only acceptable overloads are those for which we can answer, "No". In practice this usually means creating an "environment" using Internal`InheritedBlock within which different semantics apply, introducing one's own symbols that are not likely to conflict with anyone else's (or even, using Module, ones that are unable in principle to cause a conflict), and writing definitions carefully so that the scope is tightly constrained and no evaluation leaks are introduced. In this spirit, here is my suggestion for how to implement, by means of the code above, the hashTable.key = value syntax (and SetDelayed and Unset at the same time, why not):

ClearAll[withHashTableSetSemantics];
withHashTableSetSemantics[expr_] :=
  Internal`InheritedBlock[{Dot},
   Unprotect[Dot];
   Dot /: Set[Dot[w_, key_], val_] /; Head[w] === hashTableWrapper :=
    w.Unevaluated[key].set[val];
   Dot /: SetDelayed[Dot[w_, key_], val_] /; Head[w] === hashTableWrapper :=
    w.Unevaluated[key].setDelayed[val];
   Dot /: Unset@Dot[w_, key_] /; Head[w] === hashTableWrapper :=
    w.Unevaluated[key].clear[];
   Protect[Dot];
   expr
  ];
SetAttributes[withHashTableSetSemantics, HoldAll];

Be aware, however, that although the semantics are close to those of Set, SetDelayed, and Unset, they aren't exactly the same because of the implementation in terms of the hash table functions themselves. Nor are they exactly the same as for the postfix form: for example, while one can write e.g. hash.(2).set@Unevaluated[1 + 1] to set an unevaluated value, the equivalent in the infix form requires two applications of Unevaluated, i.e. hash.(2) = Unevaluated@Unevaluated[1 + 1] (which is consistent with Set itself).

If, in a session, one wishes for these semantics to apply over a number of evaluations, a good way to propagate the environment is to set $Pre = withHashTableSetSemantics. This way, no global settings are modified, and the environment can be disabled again if it causes problems or when no longer needed by Unseting $Pre.

Answer 2

OK, I've found the solution. The following code works, as far as I can see without unintended side effects:

Unprotect[Dot];
Set[(h_).key:(_Symbol|_String),value_]/;MatchQ[h,_System`Utilities`HashTable]^:=
  System`Utilities`HashTableAdd[h, ToString[key], value]
h_System`Utilities`HashTable.key:(_Symbol|_String):=
  System`Utilities`HashTableGet[h, ToString[key]]
Protect[Dot];

This works as follows:

Set has attribute HoldFirst, which on one hand allows to use it on h.value despite the definition for h.value, but on the other hand inhibits evaluation of h, so it won't match _System`Utilities`HashTable if it expands to such a type. Here the /; comes into play: This causes an evaluation leak which normally is not wanted, but in this case allows us to check the resulting pattern for matching. Therefore the rule only applies if the left hand side really evaluates to a System`Utilities`HashTable.

Answer 3

In this post Leonid explains how you could get such a notation to work. Basically, you overload the Dot function using TagSet (/:).

Answer 4

This is what I'm currently using. I hope it does not have any major side effects. Any improvements welcome.

Unprotect[Dot];
SetAttributes[Dot, HoldRest];
Dot[h_, key_Symbol | key_String] := 
  System`Utilities`HashTableGet[h, ToString[Unevaluated[key]]];
Dot /: Set[Dot[h_, key_Symbol | key_String], value_] := (
   Quiet[
    System`Utilities`HashTableRemove[h, ToString[Unevaluated[key]]], 
    System`Utilities`HashTableRemove::norem];
   System`Utilities`HashTableAdd[h, ToString[Unevaluated[key]], 
    value]);