47
$\begingroup$

Leonid Shifrin once wrote (excerpted):

(Unevaluated) is one of a very few "magic symbols", along with Sequence and Evaluate - these are deeply wired into the system and can not be easily replicated or blocked, unlike Hold - in that sense, Unevaluated is more fundamental.

HoldPattern is a normal (usual) head with HoldAll attribute for the purposes of evaluation, but its magic shows in the pattern-matching: it is invisible to the pattern-matcher, and is very important ingredient of the language since it allows pattern-matcher to be consistent with the evaluation process.

David B. Wagner writes:

The symbols Evaluate, Unevaluated, and Sequence are magic cookies. This is a whimsical term used by computer scientists to refer to any type of value that has special significance to the software system of which it is a part. The behavior of these symbols is not the result of any values or attributes they possess; rather, it is "wired into" the kernel. This implies these behaviors simply cannot be altered, nor can they be duplicated.

These are fundamental to the language but they are not widely recognized as such. I have never attempted to make an exhaustive list of these "magic Symbols" but I think it would be useful to the community to have one.

What are all the "magic" Symbols in the Mathematica language?

What makes each of these Symbols special?

  • I believe these should be considered distinct from atomic objects, which while also being "deeply wired into the system" and "not easily replicated" are a form of data types with special handling.

A related question with a somewhat different scope:

Improve this question
$\endgroup$
16
  • 3
    $\begingroup$ Oh, gotcha! You mean it's writable, but in contrast, Unevaluate or so is simply CANNOT be written directly. $\endgroup$
    – Wjx
    Aug 2, 2016 at 11:25
  • 3
    $\begingroup$ Would you consider things like Catch/Throw or Check/CheckAbort/CheckAll to fall into the "magic" category? They pretty much need special kernel hooks to function correctly, but it feels like a different level of magic than Unevaluated etc. $\endgroup$
    – WReach
    Aug 2, 2016 at 17:29
  • 3
    $\begingroup$ @WReach - I think Catch and Throw could be written - they're non trivial and would involve Catch rewriting the entire expression given as its first argument - but LISP macros are written do that kind of thing a lot and I don't think there's anything special about them here. I believe the Check family can be written as well. (For the curious, Let Over Lambda is probably the best book on LISP macros ... ever!) $\endgroup$
    – davidbak
    Aug 3, 2016 at 0:14
  • 1
    $\begingroup$ @davidbak It is exactly this kind of discussion/information that I hoped this question would inspire. Thank you. (I know this site isn't supposed to be for discussions per se, but discussion for the purpose of developing questions or answers is always welcome.) It sounds a little strange but it might be good to start an answer to list things that are not "magic" Symbols though they at first appear so, with explanations. $\endgroup$
    – Mr.Wizard
    Aug 3, 2016 at 5:21
  • 2
    $\begingroup$ Would you consider Update as magic? $\endgroup$
    – Greg Hurst
    Mar 26, 2021 at 14:58

3 Answers 3

Reset to default
28
$\begingroup$

uh oh, another "work in progress" answer from me that may never be finished.

Robby Villegas, Working with Unevaluated Expressions:

Unevaluated is not meant to be a function or stable data type. It is to be used as a wrapper on an argument in stage 1, before argument evaluation. It is a signal to the evaluator to suppress the usual evaluation of that argument.

It is transparent to the function receiving the argument. You can think of it as a shuttle giving the argument safe transport to the function's code, keeping the evaluator away. Unevaluated vanishes before the argument is fed to the function, since its purpose is fulfilled.

As Szabolcs wrote:

I'll also note that Function is a very special symbol that has very special evaluation rules: the evaluator will hold y in an expression of the form f[x][y] only when f is Function (and in no other situation). In particular, it is not possible to bestow a custom symbol with this property.

  • Pattern elements

Arguably a distinct class, these Symbols are "magic" only in pattern matching and not main evaluation itself. Robby Villegas describes these as having "meta-expression status."

Via a method from jkuczm, May 8 '15

    {Alternatives, Blank, BlankNullSequence, BlankSequence, Condition, Except,
    HoldPattern, IgnoringInactive, Literal, Longest, Optional, OptionsPattern,
    OrderlessPatternSequence, Pattern, PatternSequence, PatternTest, Repeated,
    RepeatedNull, Shortest, Verbatim}

Some nonstandard evaluation appears to take place here.

undetermined

These might have limited "magic" behavior in one form or another.

It is not immediately apparent to me how to fully emulate the partial evaluation used on the left hand side of Set; perhaps special evaluation methods are used. Still this probably doesn't belong here.

These also probably don't belong here but they certainly are unusual.

  • Attributes, like Orderless, Temporary, Listable, NHoldAll, etc., cannot be re-implemented in top-level code robustly. They would seem to fit the description of "... any type of value that has special significance to the software system of which it is a part."
Improve this answer
$\endgroup$
13
  • 4
    $\begingroup$ There is also Nothing. It is like Sequence[], but it only gets removed from lists. $\endgroup$
    – Szabolcs
    Aug 2, 2016 at 9:51
  • $\begingroup$ @Szabolcs Unfortunately it is not present in the version I use therefore I cannot examine its behavior, or definition if such exists. Have you looked into its mechanism at all? $\endgroup$
    – Mr.Wizard
    Aug 2, 2016 at 9:52
  • 2
    $\begingroup$ @Szabolcs What exactly is magical there? Common use case can be coded e.g.: not /: {a___, not, b___} := {a, b} $\endgroup$
    – Kuba ♦
    Aug 2, 2016 at 9:53
  • $\begingroup$ @Kuba Okay, I just reviewed that section. I don't see anything that really fits the description of "magic" quoted in the question, e.g. While should be easy to emulate. Trace does things "we" cannot but I don't think it really fits here, though I forgot about it or I might have grouped it with Stack. $\endgroup$
    – Mr.Wizard
    Aug 2, 2016 at 10:07
  • $\begingroup$ Maybe it isn't special then. $\endgroup$
    – Szabolcs
    Aug 2, 2016 at 10:12
13
$\begingroup$

[This response is intended as a "big list" contribution rather than a comprehensive treatment.]

Program Control Symbols

If we define "magical symbol" to mean a symbol whose implementation cannot reasonably be achieved in user code, then the various program control symbols would almost certainly qualify.

Language Features:

While it is possible in principle to implement the language features in user code through clever code transformation, the effort amounts to replacing the built-in evaluator completely. The reason is that Mathematica supports evaluation of code generated at run-time. Static analysis of the code is not sufficient to identify all evaluatable code -- the program must actually be run. Thus, the code transformer would have to take over the full evaluation process in order to accomplish its task. The implementation effort would be larger than writing the built-in evaluator in the first place -- and the program control symbols would be "magical" in the rewritten evaluator.

Even Common Lisp, with far more well-defined evaluation semantics than Mathematica, was forced to provide throw, catch, go and unwind-protect as special forms for the same reasons.

Some function interact with the evaluator:

These are somewhat less magical than the language features, but they do require access to internal state of the evaluation implementation.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I believe your description of Common Lisp's implementation of throw and catch is incorrect. They are "special forms" because catch has an implicit progn. "Special form", in Common Lisp, does mean "special evaluation" but in this case it is no more than not evaluating all arguments before the call. (In other words, it is the HoldAll attribute.) It could have been provided as a macro. (And perhaps is, in some implementations.) The reason these things are harder in Mathematica than Common Lisp (or Scheme) is that the Mathematica evaluator has a lot more kinds of "special form". $\endgroup$
    – davidbak
    Aug 4, 2016 at 2:56
  • $\begingroup$ @davidbak I think if it were possible to implement catch as a macro, it would have been. That was a driving goal in the design of Common Lisp: to reduce the set of special forms to the absolute minimum possible. $\endgroup$
    – WReach
    Aug 4, 2016 at 14:11
1
$\begingroup$

The behavior of GeoGraphics`$GeoBackgroundType (I think introduced in V12.2) is quite unusual and might not be readily reproducible top-level.

This is an internal flag to toggle between vector & raster geo data. It's use is explained around the 8:00 mark here.

It's use:

GeoGraphics[]


GeoGraphics`$GeoBackgroundType = "Legacy";

GeoGraphics[]

The strange behavior is with Unset and Clear:

GeoGraphics`$GeoBackgroundType=.

GeoGraphics`$GeoBackgroundType

"Hybrid"

Clear[GeoGraphics`$GeoBackgroundType]

During evaluation of In[9]:= 
    Clear::spsym: Special symbol GeoGraphics`$GeoBackgroundType cannot be cleared.

GeoGraphics`$GeoBackgroundType

"Hybrid"
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.