88
$\begingroup$

I wish to make a replacement inside a held expression:

f[x_Real] := x^2;
Hold[{2., 3.}] /. n_Real :> f[n]

The desired output is Hold[{4., 9.}], but I get Hold[{f[2.], f[3.]}] instead. What is the best way to make such a replacement without evaluation of the held expression?

$\endgroup$

migrated from stackoverflow.com Jul 28 '13 at 6:31

This question came from our site for professional and enthusiast programmers.

80
$\begingroup$

Generally, you want the Trott-Strzebonski in-place evaluation technique:

f[x_Real]:=x^2;
Hold[{Hold[2.],Hold[3.]}]/.n_Real:>With[{eval = f[n]},eval/;True]

(* Hold[{Hold[4.],Hold[9.]}] *)

It will inject the evaluated r.h.s. into an arbitrarily deep location in the held expression, where the expression was found that matched the rule pattern. This is in contrast with Evaluate, which is only effective on the first level inside Hold (won't work in the example above). Note that you may evaluate some things and not evaluate others:

g[x_] := x^3;
Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = f[n]}, g[eval] /; True]

(* Hold[{Hold[g[4.]], Hold[g[9.]]}] *)

The basic idea is to exploit the semantics of rules with local variables shared between the body of With and the condition, but within the context of local rules. Since the condition is True, it forced the eval variable to be evaluated inside the declaration part of With, while the code inside the Condition , here the body of With (g[eval]), is treated then as normally the r.h.s. of RuleDelayed is. It is important that With is used, since it can inject into unevaluated expressions. Module and Block also have the shared variable semantics, but wouldn't work here: while their declaration part would evaluate, they would not be able to communicate that result to their body that remains unevaluated (more precisely, only the part of the body that is inside Condition will remain unevaluated - see below). The body of With above was not evaluated either, however With injects the evaluated part ( eval here) into it - this is why the g function above remained unevaluated when the rule applied. This can be further illustrated by the following:

Hold[{Hold[2.],Hold[3.]}]/.n_Real:>Module[{eval=f[n]},
   With[{eval = eval},g[eval]/;True]]

(* Hold[{Hold[g[4.]],Hold[g[9.]]}] *)

Note b.t.w. that only the part of code inside With that is inside Condition is considered a part of the "composite rule" and therefore not evaluated. So,

Hold[{Hold[2.],Hold[3.]}]/.n_Real:>Module[{eval = f[n]},
    With[{eval = eval},Print[eval];g[eval]/;True]]

(* print: 4. *)
(* print: 9. *)
(* Hold[{Hold[g[4.]],Hold[g[9.]]}] *)

But

Hold[{Hold[2.],Hold[3.]}]/.n_Real:>Module[{eval = f[n]},
   With[{eval = eval},(Print[eval];g[eval])/;True]]

(* Hold[{Hold[Print[4.];g[4.]],Hold[Print[9.];g[9.]]}] *)

This should further clarify this mechanism.

$\endgroup$
  • 4
    $\begingroup$ The Trott-Strzebonski method looks a bit magical. Could you please explain how it works? $\endgroup$ – Alexey Popkov Jul 9 '11 at 8:18
  • 1
    $\begingroup$ @Alexey I updated my post with some explanation. It may not be completely satisfactory, but this is how I think of it currently. $\endgroup$ – Leonid Shifrin Jul 9 '11 at 10:54
  • $\begingroup$ Why Condition inside With forces evaluation not only With (with except to the Condition) but also enclosing Module? Without the Condition the r.h.s of the rule stay completely unevaluated. $\endgroup$ – Alexey Popkov Jul 10 '11 at 4:03
  • 3
    $\begingroup$ @Alexey For the same reason as when you define a global function as f[x_]:=Module[{var = x^2},With[{var = var},Hold[var]/;var>10]]. What happens is that everything in the enclosing scoping constructs that does not go into Condition gets evaluated - this is needed to compute the result of test in Condition. The semantics of rules with shared local variables is different from the standard rule-substitution semantics, this is what makes all these things possible. The further non-triviality of Trott-Strzebonski technique is that local rules are used, so all expression levels are accessible. $\endgroup$ – Leonid Shifrin Jul 10 '11 at 7:45
  • 3
    $\begingroup$ +1 The Trott-Strzebonski technique appears to be the officially sanctioned solution. You may have some academic interest in my response which discusses a technique involving the unofficial, unsupported symbol RuleCondition. $\endgroup$ – WReach Oct 6 '11 at 20:40
75
$\begingroup$

RuleCondition provides an undocumented, but very convenient, way to make replacements in held expressions. For example, if we want to square the odd integers in a held list:

Hold[{1, 2, 3, 4, 5}] /. n_Integer :> RuleCondition[n^2, OddQ[n]]
(* Hold[{1, 2, 9, 4, 25}] *)

RuleCondition differs from Condition in that the replacement expression is evaluated before it is substituted. The second argument of RuleCondition may be omitted, defaulting to True:

Hold[{2., 3.}] /. n_Real :> RuleCondition[n^2]
(* Hold[{4., 9.}] *)

It is very unfortunate that RuleCondition has remained undocumented for so long, given its extreme usefulness. The Trott-Strzebonski trick discussed in @Leonid's answer is one way to achieve the same result using only documented symbols:

Hold[{2., 3.}] /. n_Real :> With[{eval = n^2}, eval /; True]
(* Hold[{4., 9.}] *)

A slightly less verbose technique uses Block:

Hold[{2., 3.}] /. n_Real :> Block[{}, n^2 /; True]
(* Hold[{4., 9.}] *)

Judicious use of Trace reveals that both of these techniques ultimately resolve to RuleCondition. One must make up one's mind whether it is better to use the undocumented RuleCondition or rely upon implementation artifacts in With and Block. I suspect that the behaviour is unlikely to change in all three cases as so much Mathematica code depends upon the existing behaviour.

$\endgroup$
  • 1
    $\begingroup$ Great analysis! It is unbelievable that WRI had published the Trott-Strzebonski trick but still hide this convenient function! Unbelievable but true... $\endgroup$ – Alexey Popkov Oct 6 '11 at 20:21
  • $\begingroup$ @Alexey I think that Leonid's response better qualifies as the accepted answer as the Trott-Strzebonski trick seems to be as close as one is going to come to an officially sanctioned technique. I discuss RuleCondition mainly out of academic interest. $\endgroup$ – WReach Oct 6 '11 at 20:42
  • 3
    $\begingroup$ Well. But RuleCondition is really straightforward and includes no magic and as you mentioned the Trott-Strzebonski trick relies on this function too. Probably we can use RuleCondition without risk. $\endgroup$ – Alexey Popkov Oct 7 '11 at 3:47
28
$\begingroup$

Although less magical, it can be done by ReplacePart

expr = Hold[{2, 3, 4, 5}]
pos = Position[expr, _Integer]
newparts = Extract[expr, pos] /. n_Integer :> n^2
ReplacePart[expr, Thread[pos -> newparts]]
$\endgroup$
1
$\begingroup$

Based on solutions proposed by @Leonid and @WReach I have noticed something peculiar with the use of Block and With for the Trott-Strzebonski solution.

Consider the following code:

f[x_] := x^2;
g[x_] := x^3;

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = (g@*f)[n]},
Print[eval]; eval /; True
]

this will yield:

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

using Block will give the same result as above.

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
Print@eval; eval /; True
]

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

However, things become strange - as @Leonid pointed out - with the use of Composite expression in Condition:

using With prevents the evaluation of Print

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = (g@*f)[n]},
(Print[eval]; eval) /; True
]

(* Hold[{Hold[Print[64.]; 64.], Hold[Print[729.]; 729.]}] *)

However, with the use of Block the Print statements get evaluated

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
(Print@eval; eval) /; True
] 

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

Now one begins to wonder how RuleCondition - as mentioned by @WReach - will behave with Composite expression

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> RuleCondition[Print[(g@*f)[n]]; (g@*f)[n]]

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

It turns out that RuleCondition behaves in a similar way to Block

So perhaps not a million dollar but a $5 question: Which one to use (Block, RuleCondition or With)?

In my opinion both have their own advantages. For instance, With will allow you to evaluate a part of the expression and at the same time inject unevaluated code in the substitution process.

With will enable code injection

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = (g@*f)[n]},
(g[eval]; eval) /; True
]
(* Hold[{Hold[g[64.]; 64.], Hold[g[729.]; 729.]}] *)

In the example above (in spirit with what @Leonid mentioned) we substituted the Reals and also injected code that remained unevaluated i.e. g[64] and g[729]

Block and RuleCondition together with Sow and Reap

However, if the goal is to evaluate some code while making substitution 'Block` can be quite effective:

Reap[Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
  (Sow[PrimeQ@eval]; eval) /; True
  ]][[2, 1]]
];
(* {False,False} *)


Reap[
 Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
 (Sow[EvenQ[Round@eval]]; eval) /; True]
]
(* {Hold[{Hold[64.], Hold[729.]}], {{True, False}}} *)   

In the example above we could the result whether eval generates a Prime whenever the substitution is made. Likewise, this can be achieved using RuleCondition.

$\endgroup$
  • 1
    $\begingroup$ Also discussed here. Note that With doesn't evaluate its second argument at all (it becomes evaluated after leaving With), while Block evaluates it. This helps to understand the difference you describe. $\endgroup$ – Alexey Popkov Dec 9 '17 at 17:30
  • $\begingroup$ @AlexeyPopkov thanks for pointing me to it ! $\endgroup$ – Ali Hashmi Dec 9 '17 at 17:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.