Set promotes Rule to RuleDelayed?

Question

In mentally building a model of how Mathematica works, I naively expected Set to create a Rule in a symbol's DownValues, and SetDelayed to create a RuleDelayed in a symbol's DownValues. Therefore, I was a bit surprised when I tested

ClearAll[g];
Set[g[5], 0];
SetDelayed[g[x_], 1];
DownValues[g] // FullForm

and saw

List[RuleDelayed[HoldPattern[g[5]],0],RuleDelayed[HoldPattern[g[Pattern[x,Blank[]]]],1]]

I naively expected to see

List[Rule[HoldPattern[g[5]],0],RuleDelayed[HoldPattern[g[Pattern[x,Blank[]]]],1]]

or even

List[Rule[g[5],0],RuleDelayed[HoldPattern[g[Pattern[x,Blank[]]]],1]]

The extra HoldPattern and RuleDelayed seem semantically harmless -- at least I wasn't able to think of a breaking example before breakfast. Looking into the issue, I find the following remarks in the documentation for HoldPattern (emphases are mine)

The left-hand sides of rules are usually evaluated, as are parts of the left-hand sides of assignments. You can use HoldPattern to stop any part from being evaluated.

The ambiguity of that statement signals that there is a whole lot more going on, here, meaning Set and SetDelayed aren't going to be exactly equivalent to the rules one would naively expect, and there must be special cases and exclusions in the works.

I'd be grateful for hints, experiences, opinions, deeper knowledge, suggested experiments, and especially pointers to clarifying documentation.

Answer 1

Both Set and SetDelayed give RuleDelayed values. So, if you do

x=y;
y=2;
Hold[x]/.OwnValues[x]

Hold[y]

That's because OwnValues evaluates to what you originally stored, and at the time you did the setting, y didn't have any values and x=y. If the values were stored with Rule, you would get, instead of

OwnValues[x]

{HoldPattern[x] :> y}

this

{HoldPattern[x] -> 2}

It surely can be stored as a Rule, but you would have a difficult time trying to look at the values without them getting evaluated into something they are not.

The (not) only difference between Set and SetDelayed can be shown in the following equivalence

x:=y

is (not) the same as

x=Unevaluated[y]

So, before storing the values, Set evaluates them, while SetDelayed doesn't. The "not" are because this is not the whole story, but the 99.9% of the story. There is the subtletly that you can't do x:={1, 2, 3}, x[[2]]=8 as it was asked in a previous question. Also, for some reason Mathematica stores the information on how you did the assignment also for DownValues and the rest, memory storage isn't exactly the same, etc. A little bit of an "advanced" mistery that you shouldn't worry about...

As to the left hand sides:

Both Rule and RuleDelayed evaluate the left hand side. So, if you want your definition to be stored (and seen when you type XValues[symbol]) as they were at definition time, then you have to store them inside HoldPattern. Thus, you can do something like

highInteger = _Integer?(# > 234 &);

f[i : highInteger] := 2 i

ClearAll@highInteger

and still have this working

f[634]

1268

As to the "as are parts of the left hand sides of assignments" part, this is in the docs. Both Set and SetDelayed evaluate their left hand sides' heads AND arguments, but don't perform a transformation of that result before doing the assignment. So,

x = y; y[2] = z;

Now, what do you think the next assignment will do?

x[2] = 98;

Prrrrrrrrrrrrrrrrrrrrrrrrrr.....

DownValues@y

{HoldPattern[y[2]] :> 98}

DownValues@x

{}

DownValues@z

{}