# What are all the "magic" Symbols in the Mathematica language?

Leonid Shifrin once wrote (excerpted):

(Unevaluated) is one of a very few "magic symbols", along with Sequence and Evaluate - these are deeply wired into the system and can not be easily replicated or blocked, unlike Hold - in that sense, Unevaluated is more fundamental.

HoldPattern is a normal (usual) head with HoldAll attribute for the purposes of evaluation, but its magic shows in the pattern-matching: it is invisible to the pattern-matcher, and is very important ingredient of the language since it allows pattern-matcher to be consistent with the evaluation process.

David B. Wagner writes:

The symbols Evaluate, Unevaluated, and Sequence are magic cookies. This is a whimsical term used by computer scientists to refer to any type of value that has special significance to the software system of which it is a part. The behavior of these symbols is not the result of any values or attributes they possess; rather, it is "wired into" the kernel. This implies these behaviors simply cannot be altered, nor can they be duplicated.

These are fundamental to the language but they are not widely recognized as such. I have never attempted to make an exhaustive list of these "magic Symbols" but I think it would be useful to the community to have one.

What are all the "magic" Symbols in the Mathematica language?

What makes each of these Symbols special?

• I believe these should be considered distinct from atomic objects, which while also being "deeply wired into the system" and "not easily replicated" are a form of data types with special handling.

A related question with a somewhat different scope:

• Oh, gotcha! You mean it's writable, but in contrast, Unevaluate or so is simply CANNOT be written directly.
– Wjx
Aug 2 '16 at 11:25
• Would you consider things like Catch/Throw or Check/CheckAbort/CheckAll to fall into the "magic" category? They pretty much need special kernel hooks to function correctly, but it feels like a different level of magic than Unevaluated etc. Aug 2 '16 at 17:29
• @WReach - I think Catch and Throw could be written - they're non trivial and would involve Catch rewriting the entire expression given as its first argument - but LISP macros are written do that kind of thing a lot and I don't think there's anything special about them here. I believe the Check family can be written as well. (For the curious, Let Over Lambda is probably the best book on LISP macros ... ever!) Aug 3 '16 at 0:14
• @davidbak It is exactly this kind of discussion/information that I hoped this question would inspire. Thank you. (I know this site isn't supposed to be for discussions per se, but discussion for the purpose of developing questions or answers is always welcome.) It sounds a little strange but it might be good to start an answer to list things that are not "magic" Symbols though they at first appear so, with explanations. Aug 3 '16 at 5:21
• Would you consider Update as magic? Mar 26 at 14:58

uh oh, another "work in progress" answer from me that may never be finished.

• ### Unevaluated

Robby Villegas, Working with Unevaluated Expressions:

Unevaluated is not meant to be a function or stable data type. It is to be used as a wrapper on an argument in stage 1, before argument evaluation. It is a signal to the evaluator to suppress the usual evaluation of that argument.

It is transparent to the function receiving the argument. You can think of it as a shuttle giving the argument safe transport to the function's code, keeping the evaluator away. Unevaluated vanishes before the argument is fed to the function, since its purpose is fulfilled.

• ### Function

As Szabolcs wrote:

I'll also note that Function is a very special symbol that has very special evaluation rules: the evaluator will hold y in an expression of the form f[x][y] only when f is Function (and in no other situation). In particular, it is not possible to bestow a custom symbol with this property.

• ### Pattern elements

Arguably a distinct class, these Symbols are "magic" only in pattern matching and not main evaluation itself. Robby Villegas describes these as having "meta-expression status."

Via a method from jkuczm, May 8 '15

    {Alternatives, Blank, BlankNullSequence, BlankSequence, Condition, Except,
HoldPattern, IgnoringInactive, Literal, Longest, Optional, OptionsPattern,
OrderlessPatternSequence, Pattern, PatternSequence, PatternTest, Repeated,
RepeatedNull, Shortest, Verbatim}

• ### OptionsPattern / OptionValue

Some nonstandard evaluation appears to take place here.

### undetermined

These might have limited "magic" behavior in one form or another.

It is not immediately apparent to me how to fully emulate the partial evaluation used on the left hand side of Set; perhaps special evaluation methods are used. Still this probably doesn't belong here.

These also probably don't belong here but they certainly are unusual.

• There is also Nothing. It is like Sequence[], but it only gets removed from lists. Aug 2 '16 at 9:51
• @Szabolcs Unfortunately it is not present in the version I use therefore I cannot examine its behavior, or definition if such exists. Have you looked into its mechanism at all? Aug 2 '16 at 9:52
• @Szabolcs What exactly is magical there? Common use case can be coded e.g.: not /: {a___, not, b___} := {a, b}
– Kuba
Aug 2 '16 at 9:53
• @Kuba Okay, I just reviewed that section. I don't see anything that really fits the description of "magic" quoted in the question, e.g. While should be easy to emulate. Trace does things "we" cannot but I don't think it really fits here, though I forgot about it or I might have grouped it with Stack. Aug 2 '16 at 10:07
• Maybe it isn't special then. Aug 2 '16 at 10:12

[This response is intended as a "big list" contribution rather than a comprehensive treatment.]

Program Control Symbols

If we define "magical symbol" to mean a symbol whose implementation cannot reasonably be achieved in user code, then the various program control symbols would almost certainly qualify.

Language Features:

While it is possible in principle to implement the language features in user code through clever code transformation, the effort amounts to replacing the built-in evaluator completely. The reason is that Mathematica supports evaluation of code generated at run-time. Static analysis of the code is not sufficient to identify all evaluatable code -- the program must actually be run. Thus, the code transformer would have to take over the full evaluation process in order to accomplish its task. The implementation effort would be larger than writing the built-in evaluator in the first place -- and the program control symbols would be "magical" in the rewritten evaluator.

Even Common Lisp, with far more well-defined evaluation semantics than Mathematica, was forced to provide throw, catch, go and unwind-protect as special forms for the same reasons.

Some function interact with the evaluator:

These are somewhat less magical than the language features, but they do require access to internal state of the evaluation implementation.

• I believe your description of Common Lisp's implementation of throw and catch is incorrect. They are "special forms" because catch has an implicit progn. "Special form", in Common Lisp, does mean "special evaluation" but in this case it is no more than not evaluating all arguments before the call. (In other words, it is the HoldAll attribute.) It could have been provided as a macro. (And perhaps is, in some implementations.) The reason these things are harder in Mathematica than Common Lisp (or Scheme) is that the Mathematica evaluator has a lot more kinds of "special form". Aug 4 '16 at 2:56
• @davidbak I think if it were possible to implement catch as a macro, it would have been. That was a driving goal in the design of Common Lisp: to reduce the set of special forms to the absolute minimum possible. Aug 4 '16 at 14:11