How to avoid nested With[]?

Question

With[
  {v1 = #},
  With[
    {v2 = f[v1]},
    g[v1, v2]
  ]
]

How to avoid nested With[] like the above? I'd like to use v1 and v2=f[v1] in the module's body. Is using Module[{v1, v2}, v2=f[v1]; g[v1, v2]] the best/only way to avoid nested module?

Answer 1

I don't think one can avoid the need for nested With altogether - I find it a very common case to need declared variables use previously declared variables.

Since I once wrote the function (actually macro) that automates nesting With, and generates nested With at run-time, this is a good opportunity to (re)post it as an answer to an exact question that it actually addresses. I will partly borrow the discussion from this answer.

Implementation

Edit Aug.3, 2015 - added RuleDelayed UpValue, per @Federico's suggestion

Here is the code for it (with added local-variable highlighting):

ClearAll[LetL];
SetAttributes[LetL, HoldAll];
SyntaxInformation[LetL] = {
   "ArgumentsPattern" -> {_, _}, 
   "LocalVariables" -> {"Solve", {1, Infinity}}
};
LetL /: (assign : SetDelayed | RuleDelayed)[
  lhs_,rhs : HoldPattern[LetL[{__}, _]]
] := 
  Block[{With}, 
    Attributes[With] = {HoldAll}; 
    assign[lhs, Evaluate[rhs]]
  ];
LetL[{}, expr_] := expr;
LetL[{head_}, expr_] := With[{head}, expr];
LetL[{head_, tail__}, expr_] := 
  Block[{With}, Attributes[With] = {HoldAll};
    With[{head}, Evaluate[LetL[{tail}, expr]]]];

What it does is to first expand into a nested With, and only then allow the expanded construct to evaluate. It also has a special behavior when used on the r.h.s. of function definitions performed with SetDelayed.

I find this macro interesting for many reasons, in particular because it uses a number of interesting techniques together to achieve its goals (UpValues, Block trick, recursion, Hold-attributes and other tools of evaluation control, some interesting pattern-matching constructs).

Simple usage

First consider simple use cases such as this:

LetL[{a=1,b=a+1,c=a+b+2},{a,b,c}]

{1,2,5}

We can trace the execution to see how LetL expands into nested With:

Trace[LetL[{a=1,b=a+1},{a,b}],_With]

{{{{With[{b=a+1},{a,b}]},With[{a=1},With[{b=a+1},{a,b}]]},
    With[{a=1},With[{b=a+1},{a,b}]]},
    With[{a=1},With[{b=a+1},{a,b}]],With[{b$=1+1},{1,b$}]} 

Definition-time expansion in function's definitions

When LetL is used to define a function (global rule) via SetDelayed, it expands not at run-time, but at definition-time, having overloaded SetDelayed via UpValues. This is essential to be able to have conditional global rules with variables shared between the body and the condition semantics. For a more detailed discussion of this issue see the linked above answer, here I will just provide an example:

Clear[ff];
ff[x_,y_]:= LetL[{xl=x,yl=y+xl+1},xl^2+yl^2/;(xl+yl<15)];
ff[x_,y_]:=x+y;

We can now check the definitions of ff:

?ff

Global`ff
 ff[x_,y_]:=With[{xl=x},With[{yl=y+xl+1},xl^2+yl^2/;xl+yl<15]]

ff[x_,y_]:=x+y

Now, here is why it was important to expand at definition time: had LetL always expanded at run time, and the above two definitions would be considered the same by the system during definition time (variable-binding time), because the conditional form of With (also that of Module and Block) is hard-wired into the system; inside any other head, Condition has no special meaning to the system. The above-mentioned answer shows what happens with a version of Let that expands at run time: the second definition simply replaces the first.

Remarks

I believe that LetL fully implements the semantics of nested With, including conditional rules using With. This is so simply because it always fully expands before execution, as if we wrote those nested With constructs by hand. In this sense, it is closer to true macros, as they are present in e.g. Lisp.

I have used LetL in a lot of my own applications and it never let me down. From my answers on SE, its most notable presence is in this answer, where it is used a lot and those uses illustrate its utility well.

Answer 2

Introduced in V10.4 or earlier, but after V10.1

This functionality has snuck into With (ref: Daniel's comment). Note the use of the braces.

With[{v1 = #}, {v2 = f[v1]}, g[v1, v2]]
(*  g[#1, f[#1]]  *)

The syntax coloring has not caught up yet:

In V10 --

Needs["GeneralUtilities`"];
?GeneralUtilities`Where

Where[ass1, ass2, ..., expr] is a version of With that supports multiple sequential assignments.

Needs["GeneralUtilities`"];
Where[v1 = #, v2 = f[v1], g[v1, v2]]

(* g[#1, f[#1]] *)

Where[x = 2, t = x^2, Hold[x + t]]

(* Hold[2 + 4] *)

Answer 3

With works by performing a substitution operation prior to executing its body, and likely it is only a single pass. So, inter-referencing the variables is not possible. Since With accepts the use of SetDelayed (:=), you might think that that could be used, instead. For example,

With[{v1 = #, v2 := f[v1]}, g[v1, v2]]& @ p
(* g[p, f[v1]] *)

which reveals the other use of With: localization. The v1 in f[v1] is not the same as the v1 used by With, so that method is out, also. The same problem exists for Module as it uses a similar form of localization.

However, Block works

Block[{v1 = #, v2 = f[v1]}, g[v1, v2]]& @ p
(* g[p, f[p]] *)

even though the syntax highlighting makes it appear that the v1 inside f is not localized. But, Block has the attribute HoldAll, so v2 = f[v1] is not executed until v1 has taken on its local definition, and, unlike in With and Module, it is not internally treated with Unique[v1].

Answer 4

Perhaps this will work:

SetAttributes[BetterWith, HoldAll]
BetterWith[{x_}, expr_] := With[{x}, expr];
BetterWith[{x_, rest__}, expr_] := BetterWith[{x}, BetterWith[{rest}, expr]]
BetterWith[{s:Verbatim[Set][x_List, y_], rest___}, expr_] := Quiet @ With[
        {x2 = Replace[Hold[x], z_Symbol :> Pattern[z, _], {2}]},
            Replace[y, {Apply[HoldPattern,x2] :> BetterWith[{rest}, expr], _ -> $Failed}]
] 

BetterWith[{y=x+1,x=1},y]     
(* gives 2 *)

Answer 5

As of 10.4

Where appears to no longer reside in "GeneralUtilities'" however I quite like its form so here is the <= 10.3 Definition (v. similar to M.R.'s answer) that can be placed in an init.m file.

SetAttributes[Where, HoldAll];

Where[s : Verbatim[Set][x_List, y_], rest___, expr_] := 
With[{x2 = Quiet[Replace[Hold[x], z_Symbol :> z_, {2}]]}, 
Replace[y, {HoldPattern @@ x2 :> Where[rest, expr], _ -> $Failed}]]

Where[expr_] := expr

Where[x_List, expr_] := With[x, expr]

Where[x_, expr_] := With[{x}, expr]

Where[x_, rest__, expr_] := Where[x, Where[rest, expr]]

Note that this slightly improves the previous "GeneralUtilities'" definition by giving effect to a "multiple set" presumably originally intended by the first Where[s:Verbatim[Set] ... definition:

Where[
      {x, y} = {1, 3 + x},
       z = 2,
      {x, y, z}] 

(* {1, 3 + x, 2} *)

whereas

In 10.3.1

Needs["GeneralUtilities`"];

Where[
      {x, y} = {1, 3 + x},
       z = 2,
      {x, y, z}]

(* 
 With::lvset: Local variable specification {{x,y}={1,3+x}} contains {x,y}={1,3+x}, which is an assignment to {x,y}; only assignments to symbols are allowed. >>

 With[{{x, y} = {1, 3 + x}}, Where[z = 2, {x, y, z}]]

*)

Update - turns out Where can (if rarely) let a few bullets through; To collate:

First, some compression helping:

SetAttributes[CompressCode,HoldAll];
CompressCode[code_] := Compress@Hold@code;
UncompressCode[compressed_] := ReleaseHold@Uncompress@compressed;
UncompressCode[compressed_List] := Scan[UncompressCode, compressed];

i.e. with the compressed strings coming from CompressCode[(* defs *)] with defs being substituted by the code that defines Where, LetL and Let (the latter two due to Leonid)

WhereDef = "1:eJytVNtOhDAQZUFdo9kH/QMTv8A/wFt8MEZ3E/e5SMk2Foq0TcCvt9NSunRhZY0vE2jPnDlnOu1VwpbZLAgCfqTCE6NpNoe/SxXuWF4yWaQPdVlhzgkrshD2FiqssIiFqEgihdo6VivrDVaoecsSU2rAZwZ8jylqcJpFsNbBDQRyXpEQuCq4lqKXjapTFd5xlSBBch4ZruGs2iQA9y1Fxac29Ez4CLzJgh58FwX5ypEwwAsLfJGUrvCXxMXHgH5Iwqpfg+ywuSZi4xru9LXeOPzUN87LmyTKcWTrLHFJkSrcO7LWv+Y5B5Ck2DZ80Py336sTqN7kCaNjCZ6fngOizyv0NLZtDne9DmoE7rgsaaO3wZYV0LakQ5lJs8fjWt5VAm5PsBZ2/YgItfW8wZwF+wdz5GC96h5p+AvpgXM7abgM6X/omnJBDpz1KeKmPBHTxLnbu7DAP95cN3ZbV23PLOoPeCl+APfNJQs=";

LetLDef = "1:eJzNk9tOAkEMhhfxFBGjvoGPgkiiCTGGNXpdoAsTygzOzBJ4ezs7e3APuMErbybtTvu3/ab7MFWTqBMEgTnl41nRPLp03j0fQ7XeqFjOR7uNRmOEkj7SBQwJQQ+IkrQx2nF04q76fIRoB9ZqMY0tZ2X35iLV5yQf2/WxXvPOOXtpYfciI6XXYLlcRTzxhLGFN4kJw1s2BnoRr1Fa8wbWopaVhMB5Z3w8EshV1S3L3bgkNQP6AC1gSmjKYqFLDBVtsfxd5Gx4AiGF3UfdDMk7LHjSJyTY47w8lRdxcNLWzTnbwLgX6RTXbj5yVwxly0ivPLhMrpd2nvp1QdcGLU3j2NU4zXHJIL30uepAXeedoIa3nwmH+BWjnGFjPe+q2aoskRifwi591IHtq6+NQ1FZtkSlLF5bvQJxDid/vdEWKAZWy3CcZIUK5hUaQX07f3J1N8hTVIAU39srdNorLBHmrU98oJWiWgO9TPyYfhv+1z/3a0HQoRU7bsD/sXwtmOub+Nsu5ITyyRPjNSb6BgoRTRw=";

LetDef = "1:eJzVVslu2zAQlZduaXto/6Cf4qQJ2iIpUjtNzrQ5ioXQlCpRQfz35XA1aUqyAaNAL4REcRa+92ZGX5blPB9lWdZM5fKtZDR/j2+f5XJRbqqy5fTypaqhaYqS52P89lEuCxAzIepi2Qr56UzuVKQWd+saCG3eGFczxrTFmbb4Coxsgfo9b6P30PCWCAE1bzAplme4/Uku54zwp58tYwv40wJfwb7FRD7X60bbvLI2+hy+zqqKbdU9f5QFzyfWfg4VI9YhpnVDqu+cwotMVUHzVi5XLV8JB8Fr7Q449eAtWCkKfNNbGPGqqBsRnRi7E+jE3D5kwL1dF40wQKiwSM28ZRAA6cySEKbhiE8V8anRvueJJtF4daFvJYcWeuMryF9DMrY4XjAgtRSGMrkGbcpAdEjLx1FYREaB0ibWwR159GKzJrnHHdO4h3pJRLFJSjMWFesQVUp8I8sTpmU+evwwkczmqTw5OXep9pyVq6dIEvjwUIi1PtVXq4Y0bY933ala5yV0PlC9DhGH5eUzYS0ROwVkKsrSjLgk9dsNlsst280tLd5lhJ3Z7Ow8u4oInGEYUj82R3UdNFqWdBsTOLWhGIgEfRgnsnW3TNw5ukfgfACirgQTMSP5pUnwDgczG+9nFgiyn9fjeOjr/jEcskTESUhOZkFhdVAWzXaTThcHw01JZaEkhCNgU6UqLTVVNYyRWbII+4cI7QMrYHzSybjLIhgW1PHhMLYYeow6K/ldhp2P00JN5kOllyRNBjq4x/9TiQ0UytSA1RfrABdeiz3pql+aGuCXb9y/OZjuT8MR+EEuM4b+5Zh9NtNGRRlojIEWovHvLhsIy9AXjprARsVNDRtH/WArOwDDk42Ooyj/n/4kOs65C+3/UpxqiqoHZOMvo9XPxg==";

and now some defining:

UncompressCode@{WhereDef, LetLDef, LetDef}

we get the following syntactic variants (ordered by rough chronology) that all do the same thing:

With[{x = 1}, With[{y = x}, {x, y}]]
LetL[{x = 1, y = x}, {x, y}]
Where[x = 1, y = x, {x, y}]
Let[x = 1, y = x, {x, y}]
With[{x = 1}, {y = x}, {x, y}]

(* {1,1}  *)

(* For all of the above*)

Where and Let also permit threaded List Assignments

Where[{x, y} = {1, 1}, {x, y}]
Let[{x, y} = {1, 1}, {x, y}]

(* {1,1}  *)

(* For both*)

while noting users' potential scoping oversights

Where[{x, y} = {1, x}, {x, y}]
Let[{x, y} = {1, x}, {x, y}]

(* {1,x}  *)

(* For both*)

(but which I suspect differ in more involved scenarios that suggest Let is more robust)

I think it is important to have a succinct, natural, robust and efficient version of this scoping construct given its ubiquity. My vote would be for Let based on how it satisfies the first two qualities while seeming to do likewise for the latter two.

Answer 6

Let's create nested With with knowledge about RawBoxes. I'd not use this method in this case, due to the performance but it is a good exercise:

SetAttributes[myWith, HoldAll];

myWith[{spec___}, body_] := ToExpression @@ Fold[
  RawBoxes @ MakeBoxes[With[{#2}, #]] &, 
  RawBoxes @ MakeBoxes @ body, 
  RawBoxes /@ MakeBoxes /@ Unevaluated[{spec}] // Reverse
]

It is handy to nest with RawBoxes but we have to get rid of the very outer RawBoxes at the end. Notice that I'm doing this with ToExpression @@ which also make expression from what is left:

a = b = 4;
myWith[{a = 1, b = a + 1}, {a + b, Hold[a, b]}]
a
b

{3, Hold[1, 2]}
4
4