90
$\begingroup$
With[
  {v1 = #},
  With[
    {v2 = f[v1]},
    g[v1, v2]
  ]
]

How to avoid nested With[] like the above? I'd like to use v1 and v2=f[v1] in the module's body. Is using Module[{v1, v2}, v2=f[v1]; g[v1, v2]] the best/only way to avoid nested module?

$\endgroup$
2

6 Answers 6

73
$\begingroup$

I don't think one can avoid the need for nested With altogether - I find it a very common case to need declared variables use previously declared variables.

Since I once wrote the function (actually macro) that automates nesting With, and generates nested With at run-time, this is a good opportunity to (re)post it as an answer to an exact question that it actually addresses. I will partly borrow the discussion from this answer.

Implementation

Edit Aug.3, 2015 - added RuleDelayed UpValue, per @Federico's suggestion

Here is the code for it (with added local-variable highlighting):

ClearAll[LetL];
SetAttributes[LetL, HoldAll];
SyntaxInformation[LetL] = {
   "ArgumentsPattern" -> {_, _}, 
   "LocalVariables" -> {"Solve", {1, Infinity}}
};
LetL /: (assign : SetDelayed | RuleDelayed)[
  lhs_,rhs : HoldPattern[LetL[{__}, _]]
] := 
  Block[{With}, 
    Attributes[With] = {HoldAll}; 
    assign[lhs, Evaluate[rhs]]
  ];
LetL[{}, expr_] := expr;
LetL[{head_}, expr_] := With[{head}, expr];
LetL[{head_, tail__}, expr_] := 
  Block[{With}, Attributes[With] = {HoldAll};
    With[{head}, Evaluate[LetL[{tail}, expr]]]];

What it does is to first expand into a nested With, and only then allow the expanded construct to evaluate. It also has a special behavior when used on the r.h.s. of function definitions performed with SetDelayed.

I find this macro interesting for many reasons, in particular because it uses a number of interesting techniques together to achieve its goals (UpValues, Block trick, recursion, Hold-attributes and other tools of evaluation control, some interesting pattern-matching constructs).

Simple usage

First consider simple use cases such as this:

LetL[{a=1,b=a+1,c=a+b+2},{a,b,c}]
{1,2,5}

We can trace the execution to see how LetL expands into nested With:

Trace[LetL[{a=1,b=a+1},{a,b}],_With]
{{{{With[{b=a+1},{a,b}]},With[{a=1},With[{b=a+1},{a,b}]]},
    With[{a=1},With[{b=a+1},{a,b}]]},
    With[{a=1},With[{b=a+1},{a,b}]],With[{b$=1+1},{1,b$}]} 

Definition-time expansion in function's definitions

When LetL is used to define a function (global rule) via SetDelayed, it expands not at run-time, but at definition-time, having overloaded SetDelayed via UpValues. This is essential to be able to have conditional global rules with variables shared between the body and the condition semantics. For a more detailed discussion of this issue see the linked above answer, here I will just provide an example:

Clear[ff];
ff[x_,y_]:= LetL[{xl=x,yl=y+xl+1},xl^2+yl^2/;(xl+yl<15)];
ff[x_,y_]:=x+y;

We can now check the definitions of ff:

?ff
Global`ff
 ff[x_,y_]:=With[{xl=x},With[{yl=y+xl+1},xl^2+yl^2/;xl+yl<15]]

ff[x_,y_]:=x+y

Now, here is why it was important to expand at definition time: had LetL always expanded at run time, and the above two definitions would be considered the same by the system during definition time (variable-binding time), because the conditional form of With (also that of Module and Block) is hard-wired into the system; inside any other head, Condition has no special meaning to the system. The above-mentioned answer shows what happens with a version of Let that expands at run time: the second definition simply replaces the first.

Remarks

I believe that LetL fully implements the semantics of nested With, including conditional rules using With. This is so simply because it always fully expands before execution, as if we wrote those nested With constructs by hand. In this sense, it is closer to true macros, as they are present in e.g. Lisp.

I have used LetL in a lot of my own applications and it never let me down. From my answers on SE, its most notable presence is in this answer, where it is used a lot and those uses illustrate its utility well.

$\endgroup$
19
  • 3
    $\begingroup$ Out of curiosity, why did you call it "LetL" ? $\endgroup$
    – QuantumDot
    Jul 26, 2014 at 21:14
  • 2
    $\begingroup$ @QuantumDot The Let part comes from an analogy with Lisp, where there is a similar function. The L stands for Leonid, because originally there was a mathgroup thread with different implementations, and I had to somehow differentiate mine from the others. $\endgroup$ Jul 27, 2014 at 10:08
  • 3
    $\begingroup$ Why wrap Verbatim around SetDelayed? $\endgroup$
    – bdforbes
    Jun 23, 2015 at 2:05
  • 2
    $\begingroup$ I suggest the variant LetL /: (assign : SetDelayed | RuleDelayed)[lhs_, rhs : HoldPattern[LetL[{__}, _]]] := Block[{With}, Attributes[With] = {HoldAll}; assign[lhs, Evaluate[rhs]]];, so that one can also use it with RuleDelayed as in Range[10] /. x_Integer :> LetL[{y = x}, -y /; y <= 5]. $\endgroup$
    – Federico
    Aug 2, 2015 at 23:32
  • 1
    $\begingroup$ @masterxilo This is a problematic case indeed, and also something that no user-defined implementation can fight, because, as you noted, the outer scoping construct should be aware of the inner one to resolve the naming conflict right. That said, I would still think that this example is a pathology, rather than a typical use case. B.t.w., which extended With did you mean - the new extra built-in capabilities / syntax? $\endgroup$ Aug 23, 2016 at 17:48
51
$\begingroup$

Introduced in V10.4 or earlier, but after V10.1

This functionality has snuck into With (ref: Daniel's comment). Note the use of the braces.

With[{v1 = #}, {v2 = f[v1]}, g[v1, v2]]
(*  g[#1, f[#1]]  *)

The syntax coloring has not caught up yet:

Mathematica graphics

In V10 --

Needs["GeneralUtilities`"];
?GeneralUtilities`Where

Where[ass1, ass2, ..., expr] is a version of With that supports multiple sequential assignments.

Needs["GeneralUtilities`"];
Where[v1 = #, v2 = f[v1], g[v1, v2]]

(* g[#1, f[#1]] *)

Where[x = 2, t = x^2, Hold[x + t]]

(* Hold[2 + 4] *)
$\endgroup$
11
  • $\begingroup$ This reminds me of a Q&A I've been meaning to post for years but I never get around to. The basic code behind Where is one of the methods I was using. Maybe I'll finally get around to that soon. (+1 of course.) $\endgroup$
    – Mr.Wizard
    Jul 15, 2014 at 6:33
  • 1
    $\begingroup$ I can't get Where to work in 10.4... am I alone? $\endgroup$
    – user541686
    Mar 26, 2016 at 11:01
  • 1
    $\begingroup$ The undocumented With construct seems to be the way to go, so this should be the accepted answer IMO. It is the only variant that behaves correctly like nested Withs in the following case where LetL and Where fail: ClearAll@a; {With[{data = {a}}, With[{a = data}, With[{b = a}, {data, b}]]],With[{data = {a}},With[{a = data}, {b = a}, {data, b}]](*same result, more concise*), With[{data = {a}}, LetL[{a = data, b = a}, {data, b}]](*different result!*), Needs["GeneralUtilities`"]; With[{data = {a}}, Where[a = data, b = a, {data, b}]](*crashes because of recursion*) }. $\endgroup$
    – masterxilo
    Aug 23, 2016 at 15:49
  • 1
    $\begingroup$ @Mr.Wizard I suppose I meant the With syntax worked in V10.4 and may have worked in V10.3, and maybe in V10.2, too. I probably couldn't check versions before V10.4. Daniel's comment was that it was introduced in "V10.*something*". No one has come along and verified which version was the version in which it was first implemented. $\endgroup$
    – Michael E2
    Oct 30, 2018 at 3:23
  • 1
    $\begingroup$ @JEM_Mosig This is mentioned in Daniel's comment and it's still not fixed. I don't know of a fix nor why you can't fix it with SyntaxInformation. $\endgroup$
    – Michael E2
    Dec 30, 2019 at 15:18
30
$\begingroup$

With works by performing a substitution operation prior to executing its body, and likely it is only a single pass. So, inter-referencing the variables is not possible. Since With accepts the use of SetDelayed (:=), you might think that that could be used, instead. For example,

With[{v1 = #, v2 := f[v1]}, g[v1, v2]]& @ p
(* g[p, f[v1]] *)

which reveals the other use of With: localization. The v1 in f[v1] is not the same as the v1 used by With, so that method is out, also. The same problem exists for Module as it uses a similar form of localization.

However, Block works

Block[{v1 = #, v2 = f[v1]}, g[v1, v2]]& @ p
(* g[p, f[p]] *)

even though the syntax highlighting makes it appear that the v1 inside f is not localized. But, Block has the attribute HoldAll, so v2 = f[v1] is not executed until v1 has taken on its local definition, and, unlike in With and Module, it is not internally treated with Unique[v1].

$\endgroup$
14
  • $\begingroup$ With the potential effects of these different attributes on the program unknown, I think I'd better just use Module[{v1, v2}, v2=f[v1]; g[v1, v2]]. Thanks! $\endgroup$
    – user13253
    Sep 11, 2012 at 4:39
  • 6
    $\begingroup$ +1 for some good points. However, my comment here is that Block solves an entirely different problem. With is unique in that it creates referentially-transparent code (no assignments to its "variables" in the body of With is possible), while Block allows the body to arbitrarily modify the variables it localizes. Another big difference: (nested) With can inject into held expressions, something Block or Module can not. So, I don't really view a replacement of (nested) With with Block or Module as a generally valid solution to the posed problem. $\endgroup$ Sep 11, 2012 at 12:19
  • $\begingroup$ @LeonidShifrin absolutely. With because of its method of operation does things Block cannot do, and vice versa. But, the self-referential part, I thought, was essential to the problem. As long as the other parts of the operation of With are not needed, then Block should be a viable alternative. However, it is not a direct replacement. $\endgroup$
    – rcollyer
    Sep 11, 2012 at 12:43
  • $\begingroup$ @rcollyer Well, as long as the solution removes some defining properties of With, I would not consider it a general solution for With specifically. I would agree with you more (in that self-referential part is the main issue) if this was a question of nested Block or Module. $\endgroup$ Sep 11, 2012 at 12:54
  • $\begingroup$ @LeonidShifrin except that it is well known that you can do Scope[{x,y}, x = f[y]] with either Module or Block, so I don't think that question would come up. Personally, I think the mutability of the variables in Block is the biggest issue with using as a replacement for With. The ability to use With to inject code into held expressions is less well known. But, for the OPs problem, it was the perfect fit, provided of course the representative example doesn't stray to far from the actual code. :) $\endgroup$
    – rcollyer
    Sep 11, 2012 at 13:09
14
$\begingroup$

Perhaps this will work:

SetAttributes[BetterWith, HoldAll]
BetterWith[{x_}, expr_] := With[{x}, expr];
BetterWith[{x_, rest__}, expr_] := BetterWith[{x}, BetterWith[{rest}, expr]]
BetterWith[{s:Verbatim[Set][x_List, y_], rest___}, expr_] := Quiet @ With[
        {x2 = Replace[Hold[x], z_Symbol :> Pattern[z, _], {2}]},
            Replace[y, {Apply[HoldPattern,x2] :> BetterWith[{rest}, expr], _ -> $Failed}]
] 

BetterWith[{y=x+1,x=1},y]     
(* gives 2 *)
$\endgroup$
2
  • 1
    $\begingroup$ Just hide the nested With[] under hood of a new symbol? $\endgroup$
    – user13253
    Sep 11, 2012 at 4:37
  • 1
    $\begingroup$ @MaThEmAtika Exactly. $\endgroup$
    – M.R.
    Sep 11, 2012 at 4:42
8
$\begingroup$

As of 10.4

Where appears to no longer reside in "GeneralUtilities'" however I quite like its form so here is the <= 10.3 Definition (v. similar to M.R.'s answer) that can be placed in an init.m file.

SetAttributes[Where, HoldAll];

Where[s : Verbatim[Set][x_List, y_], rest___, expr_] := 
With[{x2 = Quiet[Replace[Hold[x], z_Symbol :> z_, {2}]]}, 
Replace[y, {HoldPattern @@ x2 :> Where[rest, expr], _ -> $Failed}]]

Where[expr_] := expr

Where[x_List, expr_] := With[x, expr]

Where[x_, expr_] := With[{x}, expr]

Where[x_, rest__, expr_] := Where[x, Where[rest, expr]]

Note that this slightly improves the previous "GeneralUtilities'" definition by giving effect to a "multiple set" presumably originally intended by the first Where[s:Verbatim[Set] ... definition:

Where[
      {x, y} = {1, 3 + x},
       z = 2,
      {x, y, z}] 

(* {1, 3 + x, 2} *)

whereas

In 10.3.1

Needs["GeneralUtilities`"];

Where[
      {x, y} = {1, 3 + x},
       z = 2,
      {x, y, z}]

(* 
 With::lvset: Local variable specification {{x,y}={1,3+x}} contains {x,y}={1,3+x}, which is an assignment to {x,y}; only assignments to symbols are allowed. >>

 With[{{x, y} = {1, 3 + x}}, Where[z = 2, {x, y, z}]]

*)

Update - turns out Where can (if rarely) let a few bullets through; To collate:

First, some compression helping:

SetAttributes[CompressCode,HoldAll];
CompressCode[code_] := Compress@Hold@code;
UncompressCode[compressed_] := ReleaseHold@Uncompress@compressed;
UncompressCode[compressed_List] := Scan[UncompressCode, compressed];

i.e. with the compressed strings coming from CompressCode[(* defs *)] with defs being substituted by the code that defines Where, LetL and Let (the latter two due to Leonid)

WhereDef = "1:eJytVNtOhDAQZUFdo9kH/QMTv8A/wFt8MEZ3E/e5SMk2Foq0TcCvt9NSunRhZY0vE2jPnDlnOu1VwpbZLAgCfqTCE6NpNoe/SxXuWF4yWaQPdVlhzgkrshD2FiqssIiFqEgihdo6VivrDVaoecsSU2rAZwZ8jylqcJpFsNbBDQRyXpEQuCq4lqKXjapTFd5xlSBBch4ZruGs2iQA9y1Fxac29Ez4CLzJgh58FwX5ypEwwAsLfJGUrvCXxMXHgH5Iwqpfg+ywuSZi4xru9LXeOPzUN87LmyTKcWTrLHFJkSrcO7LWv+Y5B5Ck2DZ80Py336sTqN7kCaNjCZ6fngOizyv0NLZtDne9DmoE7rgsaaO3wZYV0LakQ5lJs8fjWt5VAm5PsBZ2/YgItfW8wZwF+wdz5GC96h5p+AvpgXM7abgM6X/omnJBDpz1KeKmPBHTxLnbu7DAP95cN3ZbV23PLOoPeCl+APfNJQs=";

LetLDef = "1:eJzNk9tOAkEMhhfxFBGjvoGPgkiiCTGGNXpdoAsTygzOzBJ4ezs7e3APuMErbybtTvu3/ab7MFWTqBMEgTnl41nRPLp03j0fQ7XeqFjOR7uNRmOEkj7SBQwJQQ+IkrQx2nF04q76fIRoB9ZqMY0tZ2X35iLV5yQf2/WxXvPOOXtpYfciI6XXYLlcRTzxhLGFN4kJw1s2BnoRr1Fa8wbWopaVhMB5Z3w8EshV1S3L3bgkNQP6AC1gSmjKYqFLDBVtsfxd5Gx4AiGF3UfdDMk7LHjSJyTY47w8lRdxcNLWzTnbwLgX6RTXbj5yVwxly0ivPLhMrpd2nvp1QdcGLU3j2NU4zXHJIL30uepAXeedoIa3nwmH+BWjnGFjPe+q2aoskRifwi591IHtq6+NQ1FZtkSlLF5bvQJxDid/vdEWKAZWy3CcZIUK5hUaQX07f3J1N8hTVIAU39srdNorLBHmrU98oJWiWgO9TPyYfhv+1z/3a0HQoRU7bsD/sXwtmOub+Nsu5ITyyRPjNSb6BgoRTRw=";

LetDef = "1:eJzVVslu2zAQlZduaXto/6Cf4qQJ2iIpUjtNzrQ5ioXQlCpRQfz35XA1aUqyAaNAL4REcRa+92ZGX5blPB9lWdZM5fKtZDR/j2+f5XJRbqqy5fTypaqhaYqS52P89lEuCxAzIepi2Qr56UzuVKQWd+saCG3eGFczxrTFmbb4Coxsgfo9b6P30PCWCAE1bzAplme4/Uku54zwp58tYwv40wJfwb7FRD7X60bbvLI2+hy+zqqKbdU9f5QFzyfWfg4VI9YhpnVDqu+cwotMVUHzVi5XLV8JB8Fr7Q449eAtWCkKfNNbGPGqqBsRnRi7E+jE3D5kwL1dF40wQKiwSM28ZRAA6cySEKbhiE8V8anRvueJJtF4daFvJYcWeuMryF9DMrY4XjAgtRSGMrkGbcpAdEjLx1FYREaB0ibWwR159GKzJrnHHdO4h3pJRLFJSjMWFesQVUp8I8sTpmU+evwwkczmqTw5OXep9pyVq6dIEvjwUIi1PtVXq4Y0bY933ala5yV0PlC9DhGH5eUzYS0ROwVkKsrSjLgk9dsNlsst280tLd5lhJ3Z7Ow8u4oInGEYUj82R3UdNFqWdBsTOLWhGIgEfRgnsnW3TNw5ukfgfACirgQTMSP5pUnwDgczG+9nFgiyn9fjeOjr/jEcskTESUhOZkFhdVAWzXaTThcHw01JZaEkhCNgU6UqLTVVNYyRWbII+4cI7QMrYHzSybjLIhgW1PHhMLYYeow6K/ldhp2P00JN5kOllyRNBjq4x/9TiQ0UytSA1RfrABdeiz3pql+aGuCXb9y/OZjuT8MR+EEuM4b+5Zh9NtNGRRlojIEWovHvLhsIy9AXjprARsVNDRtH/WArOwDDk42Ooyj/n/4kOs65C+3/UpxqiqoHZOMvo9XPxg==";

and now some defining:

UncompressCode@{WhereDef, LetLDef, LetDef}

we get the following syntactic variants (ordered by rough chronology) that all do the same thing:

With[{x = 1}, With[{y = x}, {x, y}]]
LetL[{x = 1, y = x}, {x, y}]
Where[x = 1, y = x, {x, y}]
Let[x = 1, y = x, {x, y}]
With[{x = 1}, {y = x}, {x, y}]

(* {1,1}  *)

(* For all of the above*)

Where and Let also permit threaded List Assignments

Where[{x, y} = {1, 1}, {x, y}]
Let[{x, y} = {1, 1}, {x, y}]

(* {1,1}  *)

(* For both*)

while noting users' potential scoping oversights

Where[{x, y} = {1, x}, {x, y}]
Let[{x, y} = {1, x}, {x, y}]

(* {1,x}  *)

(* For both*)

(but which I suspect differ in more involved scenarios that suggest Let is more robust)

I think it is important to have a succinct, natural, robust and efficient version of this scoping construct given its ubiquity. My vote would be for Let based on how it satisfies the first two qualities while seeming to do likewise for the latter two.

$\endgroup$
5
$\begingroup$

Let's create nested With with knowledge about RawBoxes. I'd not use this method in this case, due to the performance but it is a good exercise:

SetAttributes[myWith, HoldAll];

myWith[{spec___}, body_] := ToExpression @@ Fold[
  RawBoxes @ MakeBoxes[With[{#2}, #]] &, 
  RawBoxes @ MakeBoxes @ body, 
  RawBoxes /@ MakeBoxes /@ Unevaluated[{spec}] // Reverse
]

It is handy to nest with RawBoxes but we have to get rid of the very outer RawBoxes at the end. Notice that I'm doing this with ToExpression @@ which also make expression from what is left:

a = b = 4;
myWith[{a = 1, b = a + 1}, {a + b, Hold[a, b]}]
a
b
{3, Hold[1, 2]}
4
4
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.