What is the story with Removed symbols?

Question

The system function Remove evidently exists primarily to turn a fatal problem into an annoying one, by giving the user a (rather blunt) instrument with which to resolve symbol clashes. Presumably, it works by removing the entry for a given symbol from whatever internal table Mathematica uses to look them up by name. However, there can be some interesting consequences when you have a reference to a symbol that you later Remove. First, we need to set up a symbol that we will remove, and give another symbol a reference to it:

In[1]:= y = 3
Out[1]= 3

In[2]:= x := y

In[3]:= x
Out[3]= 3

In[4]:= OwnValues[x]
Out[4]= {HoldPattern[x] :> y}

And then we can remove it:

In[5]:= Remove[y]

In[6]:= x
Out[6]= Removed["y"]

Now, we have Removed["y"], which is the ghost of the symbol we just bludgeoned to death. However, contrary to appearances, Removed is not the head of the expression:

In[7]:= x // Head
Out[7]= Symbol 

It's still a symbol. It appears to just have had all its values cleared when it was Remove'd. However, you need an elaborate incantation to contact a symbol that's moved on into the spirit world in order to verify this:

In[8]:= OwnValues[x]
Out[8]= {HoldPattern[x] :> Removed["y"]}

In[9]:= ReleaseHold[Hold[OwnValues[x]] /. OwnValues[x]]
Out[9]= {}

That's OK, though, because I can still give new values to the removed symbol!

In[10]:= With[{yy = x},
          yy = 17]
Out[10]= 17

In[11]:= ReleaseHold[Hold[OwnValues[x]] /. OwnValues[x]]
Out[11]= {HoldPattern[Removed[y]] :> 17}

So, what can't you do with a removed symbol that you can do with an ordinary symbol? And is there any reliable programmatic way to check whether a symbol really has been removed, seeing that it has a perfectly ordinary Symbol head? I suppose you can hack something together with StringMatchQ, but that just feels wrong and it's tough to get the evaluation order right if the removed symbol might have OwnValues.

Answer 1

Some observations on Removed

Removed is not a normal head, but rather a print form. Consider a definition

x := y

Once we Remove the y, we invalidate x in a subtle but permanent way - reintroducing the y into the session won't help. Remove is really a rather special-purpose destructuve operation, aimed more at removing auto-generated symbols. In a system of inter-connected functions (possibly from different packages), for this reason Remove is safe only if nothing depends on the symbol being removed. Resolving shadowing is the only mainstream (frequent, non-advanced) application of Remove I am aware of.

I learned from the book of David Wagner ("Power programming with Mathematica",page 251), that while the removed symbol is printed (and has a FullForm) as Removed["sym"], applying Head to it yields Symbol. The reason for that is that Removed["sym"] still represents a symbol, albeit marked for removal. For code like this:

Clear[b];b := a;Remove[a];b

you can not, e.g., "resurrect" a in b with this:

b /. Removed[x_] :> ToExpression[x]

Removed["a"]

but, since Removed["a"] represents a symbol marked for removal, this will work, to at least reconstruct the value of b:

b /. Cases[b, s_Symbol :> (s :> a), {0, Infinity}, Heads -> True]

a

You can also analyze which symbols in some expression were Remove-d, by something like this:

Clear[removedNames];
removedNames[expr_] :=
  Flatten[
    StringCases[
      ToString /@ Union@Cases[expr, _Symbol, Infinity, Heads -> True],
      ShortestMatch["Removed[" ~~ x__ ~~ "]"] :> x]
  ]

So that

removedNames[OwnValues[y]]

{"x"}

Summary

So, to summarize:

• Removed is not a normal head but a display form.
• When you Remove a symbol, Removed[sym] still represents this symbol, but being marked for removal. My understanding is that this is still a reference to the symbol table, but it becomes opaque and not bound to the symbol name any more. However, apparently, Mathematica still allows you to manipulate it, pretty much as if you manipulated the normal symbol. In particular, you can, for the purposes of pattern-matching, still count on it being a Symbol. This allows us to do some things as if the symbol in question has not been removed, particularly by using local rule substitutions.
• As you demonstrated, you can still change some ...Values for Remove-d symbols with local rules. While it looks like you can reconstruct many of the symbol's properties (except explicit symbol name), I would however limit such uses of these symbols to extreme cases when you need to make some definitions valid in a given Mathematica session.

EDIT: one constructive use - catching bugs induced by Remove

Here is one possibly constructive use of the above behavior. Remove-ing symbols is dangerous because it subtly invalidates definitions for other symbols which were referencing the symbols being removed. We may wish to know when this happens, but often the effects are silent and insidious. Here is a function which enables one to trigger such events:

ClearAll[remove];
SetAttributes[remove, HoldAll];
remove::remvd = "Stymbol `1` was removed!";
remove[s_Symbol, failCode_: Throw[$Failed, remove]] :=
  Module[{defs},
    With[{strsym = ToString[HoldForm[s]]},
       With[{body := (
          Message[remove::remvd , Style[strsym, Red]];
          failCode
           )},
        defs :=
          Module[{},
            OwnValues[s] = HoldPattern[s] :> body;
            UpValues[s] = HoldPattern[_[___, s, ___]] :> body;
          ]]];
    Remove[s];
    defs;]; 

What it does is to assign certain definitions to symbols already after they have been removed, using the behavior discussed above. These definitions execute arbitrary user-defined code when evaluation comes to the Remove-d symbol, in most cases (except when the symbols are inside HoldAllComplete heads). To augment it, here is a dynamic environment, in which Remove will behave that way:

ClearAll[withCustomRemove];
SetAttributes[withCustomRemove, HoldAll];
withCustomRemove[code_,failCode_: Throw[$Failed, remove]] :=
  Module[{inRemove},
    Internal`InheritedBlock[{Remove },
     Unprotect[Remove];
     Remove[arg_] /; ! TrueQ[inRemove] :=
         Block[{inRemove = True}, remove[arg,failCode]];
     Protect[Remove];
     code]];

and here is an example of use. First, the usual behavior:

ClearAll[f, a];
f[x_] := x + a;
g[] := Hold[a];
Remove[a];
{f[1], g[]}

{1+Removed[a],Hold[Removed[a]]}

Now, with our functions:

withCustomRemove[
   ClearAll[f, a];
   f[x_] := x + a;
   g[] := Hold[a];
   Remove[a];
]
{f[1], g[]}

  During evaluation of In[78]:= remove::remvd: Stymbol a was removed!

  During evaluation of In[78]:= Throw::nocatch: Uncaught Throw[$Failed,remove]
           returned to top level. >>

  Hold[Throw[$Failed,remove]]

One can specify a different behavior to be triggered on such an event. One can also generalize to Remove with several arguments. The idea is that you can take the code which is suspicious, and execute it inside withCustomRemove - which may often enable you to catch bugs related to attempts to use Remove-d symbols.