# Set promotes Rule to RuleDelayed?

In mentally building a model of how Mathematica works, I naively expected Set to create a Rule in a symbol's DownValues, and SetDelayed to create a RuleDelayed in a symbol's DownValues. Therefore, I was a bit surprised when I tested

ClearAll[g];
Set[g[5], 0];
SetDelayed[g[x_], 1];
DownValues[g] // FullForm


and saw

List[RuleDelayed[HoldPattern[g[5]],0],RuleDelayed[HoldPattern[g[Pattern[x,Blank[]]]],1]]


I naively expected to see

List[Rule[HoldPattern[g[5]],0],RuleDelayed[HoldPattern[g[Pattern[x,Blank[]]]],1]]


or even

List[Rule[g[5],0],RuleDelayed[HoldPattern[g[Pattern[x,Blank[]]]],1]]


The extra HoldPattern and RuleDelayed seem semantically harmless -- at least I wasn't able to think of a breaking example before breakfast. Looking into the issue, I find the following remarks in the documentation for HoldPattern (emphases are mine)

The left-hand sides of rules are usually evaluated, as are parts of the left-hand sides of assignments. You can use HoldPattern to stop any part from being evaluated.

The ambiguity of that statement signals that there is a whole lot more going on, here, meaning Set and SetDelayed aren't going to be exactly equivalent to the rules one would naively expect, and there must be special cases and exclusions in the works.

I'd be grateful for hints, experiences, opinions, deeper knowledge, suggested experiments, and especially pointers to clarifying documentation.

-
Since @Rojo already gave an extended answer, I'd just add that IMO the essential part can be summarized very briefly: rules in the rule base must be as inert as they can possibly be, so that no part of them evaluates prematurely. This is why HoldPattern is used for the l.h.s., and RuleDelayed to keep the r.h.s. also unevaluated until the replacement is made. Also, as @Rojo noted, this is not always enough, and some evaluation of the l.h.s. does happen during assignments (which was obviously an intentional design decision). – Leonid Shifrin Jun 7 '12 at 20:20
I gave a more extended analysis of evaluation during assignments here, you may find it interesting as well. – Leonid Shifrin Jun 7 '12 at 20:21

Both Set and SetDelayed give RuleDelayed values. So, if you do

x=y;
y=2;
Hold[x]/.OwnValues[x]


Hold[y]

That's because OwnValues evaluates to what you originally stored, and at the time you did the setting, y didn't have any values and x=y. If the values were stored with Rule, you would get, instead of

OwnValues[x]

{HoldPattern[x] :> y}


this

{HoldPattern[x] -> 2}


It surely can be stored as a Rule, but you would have a difficult time trying to look at the values without them getting evaluated into something they are not.

The (not) only difference between Set and SetDelayed can be shown in the following equivalence

x:=y


is (not) the same as

x=Unevaluated[y]


So, before storing the values, Set evaluates them, while SetDelayed doesn't. The "not" are because this is not the whole story, but the 99.9% of the story. There is the subtletly that you can't do x:={1, 2, 3}, x[[2]]=8 as it was asked in a previous question. Also, for some reason Mathematica stores the information on how you did the assignment also for DownValues and the rest, memory storage isn't exactly the same, etc. A little bit of an "advanced" mistery that you shouldn't worry about...

As to the left hand sides:

Both Rule and RuleDelayed evaluate the left hand side. So, if you want your definition to be stored (and seen when you type XValues[symbol]) as they were at definition time, then you have to store them inside HoldPattern. Thus, you can do something like

highInteger = _Integer?(# > 234 &);

f[i : highInteger] := 2 i

ClearAll@highInteger


and still have this working

f[634]


1268

As to the "as are parts of the left hand sides of assignments" part, this is in the docs. Both Set and SetDelayed evaluate their left hand sides' heads AND arguments, but don't perform a transformation of that result before doing the assignment. So,

x = y; y[2] = z;

Now, what do you think the next assignment will do?

x[2] = 98;

Prrrrrrrrrrrrrrrrrrrrrrrrrr.....

DownValues@y


{HoldPattern[y[2]] :> 98}

DownValues@x


{}

DownValues@z


{}

-
I just tried ClearAll[f,x];f[x_]=x^2 then the undocumented LanguageExtendedFullDefinition[f], which shows {HoldPattern[f[x_]]->x^2}, namely Rule, but DownValues[f] continues to show {HoldPattern[f[x_]]:>x^2}, rule delayed. Wheels within wheels. – Reb.Cabin Jun 7 '12 at 21:42
@Reb.Cabin, the result of that function is wrapped in LanguageDefinitionList which has attribute HoldAll, so you don't need RuleDelayed any more to allow you to see what's really stored – Rojo Jun 7 '12 at 22:34
@Reb.Cabin Very interesting, thank you. I already asked for such functionality (at least here and here). How did you find the LanguageExtendedFullDefinition function? How it differs from LanguageExtendedDefinition? – Alexey Popkov Jun 8 '12 at 19:46
@AlexeyPopkov try ??Language* -- that's how I found LanguageExtendedFullDefinition. Not sure how it differs from LanguageExtendedDefinition – Reb.Cabin Jun 10 '12 at 4:34