Manipulate in Manipulate

Question

Question 63982 (Altering values in Manipulate with dynamically generated controls) got a very interesting answer from @belisarius, in which he used a Manipulate within a Manipulate to achieve the goal. I am puzzled by his solution and will demonstrate my embarrassment by a somewhat simplified example.

In Mathematica there are actually two independent sets of variables: those that are used by the frontend and those that are used by the kernel. All interactivity in Mathematica, such as Manipulate, Button, ... is based on interaction with the user and therefore belongs completely to the domain of the frontend. But we use kernel commands with kernel variables to construct these user interfaces. Fortunately, in Mathematica it is practically never necessary to bother about the difference between frontend and kernel variables, the frontend and the kernel are very well integrated.

Let us start with a simple example, based on the question mentioned above. The button displays the current value of the two variables x and s.

Row[{Manipulate[x, {{x, s}, -1, 1}, Button[Dynamic[{x, s}], s = RandomReal[{-1, 1}]]],
  Spacer[30], Dynamic[{x, s}]}]

In the second argument of Manipulate, the s is the initial value for x. When we move the slider, we see that in the Manipulate expression the value of x is continuously updated but not outside this expression; the variable s does not change at all. That is as expected. Moreover, when we look at the input form of the Manipulate expression, we see that the current value of x is stored in the second argument of Manipulate at the position of s. That is well documentated. There is no connection at all between s and x. When we press the button, the value of both the kernel s and the frontend s changes, but is has no effect on x.

Also, when we inspect the box structure of the displayed Manipulate expression with the toggle Ctrl-E, we see a.o. the variable \$CellContext`x$$. That is, I think, the indication of a frontend variable that has no connection with a kernel variable at all. Its value is displayed on the button, but not outside the Manipulate expression. Moreover, there is a variable $CellContext`s, which, I think, is a frontend variable that is automatically linked to the kernel variable s. So we see its value both on the button and in the Dynamic output.

The following command is a simplified form of the construction of @belisarius. We wrap the Manipulate in another Manipulate:

Row[{Manipulate[Manipulate[x, {{x, s}, -1, 1}, Button[Dynamic[{x, s}], s = RandomReal[{-1, 1}]]]],
   Spacer[30], Dynamic[{x, s}]}]

When we move the slider for x, we see as before the updating on the button, but not in the Dynamic output. When we click on the button for s, of course the value of s changes, but now the value of x is set to the value of s. That means that the effect of the outer Manipulate is that by clicking on the button the inner Manipulate is completely re-evaluated, including the initialization of x to s. It might be simple to understand, but I fail to see an explanation for that.

Another effect is that in the input form we no longer see the values of x and s, and that in the box form the variable \$CellContext`x$$ is no longer is visible. Instead we have the frontend variable \$CellContext`x that is now NOT coupled to the kernel variable x.

Any hint that helps me in better understanding what is going on here is highly appreciated.

Answer 1

Let me add a simple explanation of one of your concerns. Without those context references.

Also, I don't think that FE's and Kernel's ownership of variables matters here.

According to the first section of AdvancedManipulateFunctionality:

The subject of when exactly a given dynamic expression will be updated is complex, and is addressed in "Introduction to Dynamic" and "Advanced Dynamic Functionality". In reading those, keep in mind that Manipulate simply wraps its first argument in Dynamic and passes the value of its TrackedSymbols option to a Refresh inside that. Everything to do with updating is handled by that Dynamic and Refresh.

so you can look at

Manipulate[body, {{x, s}, 0,1}]

as, more or less:

 DynamicModule[{x = s},
   Column[{
        Control[Dynamic[x], 0,1}] ,
        Dynamic[body]
        }]
 ]

so when you use your nested construction:

Manipulate[
    Manipulate[body, {{x, s}, 0,1}],
]

it is basically:

DynamicModule[{},
    Dynamic[
        DynamicModule[{x = s},
               Column[{
                    Control[Dynamic[x], 0,1}] ,
                    Dynamic[body]
                    }]
         ]
        ]
       ]

Now, when you click Button, s changes its value. And in the second example s appears inside Dynamic, so it triggers it to reevaluate the whole inner Manipulate.

I hope I have not missed the point.

Answer 2

In the mean time fortunately I started understanding the background of the problem in question 63982. I added another answer there, essentially stating that as soon as the second argument of the button does not contain the symbol c, the construction will work. The solution of @belisarius satisfies this condition. So in the sequel I will not discuss any more the problems I had with the effect of pressing the button.

In this answer on my own question above I will look at the various local variables that turn up in connection with this problem. But before doing so, I want to repeat my remark in the question: all interactivity in Mathematica belongs completely to the frontend. Now and then you can find this remark in the documentation, but mostly the documentation avoids to make any difference between frontend and kernel variables. First a short survey of some frontend variables.

Evaluate:

Button["Print x", Print[x]]

The result is a button. When used, it prints the current value of the kernel variable x. When we inspect the box structure by selecting the cell and pressing ctrl-E, we see that the frontend uses the frontend variable \$CellContext`x; such a frontend variable is automatically linked with the corresponding kernel variable mentioned after the backtick.

Now evaluate the following command. By doing so, we see that, just as with a Module, the kernel uses a temporary variable x$nnn.The resulting displayed DynamicModule is completely selfcontained and can be used in any notebook without starting the kernel. Therefore all variables that turn up in the result must be frontend variables.

DynamicModule[{x = 0.5}, 
  Print[HoldForm[x]];
  Panel[Column[{Row[{Dynamic[HoldForm[x]], " = ", Dynamic[x]}], Slider[Dynamic[x]]}]]]

Select the cell with the panel and press ctrl-E. In the box structure we see the frontend variable \$CellContext`x$$. That frontend variable is not linked to a kernel variable. If it were, it has to be x$$ and that variable is unknown to the kernel.

On the other hand, though we do not see it in the boxes, there is a kernel variable FE`x$$nnn displayed in the panel. It seems that that kernel variable is created by the frontend. Copy the panel and paste it somewhere else in the notebook, and observe that the name of the variable changes. This copying and pasting is a frontend activity only!

Why is this variable created? I think because of the frontend has to display HoldForm[x] and HoldForm is a kernel function, so it must have a kernel argument. Instead of HoldForm we can also use Unevaluated.

The kernel variable is indeed known to the kernel (adapt the number if necessary):

{FE`x$$2145, Dynamic[FE`x$$2145]}

Move the slider in the panel and observe that there is no updating. Maybe unexpected, but simple to explain: updating is only done when the kernel changed the value of a variable. Here it is the frontend. But with a special trick we can force the frontend to do a kernel evaluation by including a second argument in the Dynamic expression in the slider. We will use the default value: assign the state of the slider to the (kernel!) variable:

DynamicModule[{x = 0.5}, 
   Panel[Column[{Row[{Dynamic[Unevaluated[x]], " = ", Dynamic[x]}], 
   Slider[Dynamic[x, (x = #) &]]}]]]

In the next command, adapt the number if necessary, evaluate it and move the slider in the panel. The value is updated!

Dynamic[FE`x$$2146]

Hence it is possible to monitor the changes of a local frontend variable in a displayed DynamicModule in the kernel.

Now back to my own question, with some slightly different commands.

Manipulate[Row[{HoldForm[x]," = ", x}],{{x,s},-1,1},
  Button[Dynamic[Row[{HoldForm[s], " = ", Dynamic[s]}]], s = RandomReal[{-1,1}]]]

In the displayed Manipulate there is a local frontend variable \$CellContext`x\$\$. Since a HoldForm of this variable has to be displayed, the frontend created a special kernel variable FE`x$$nnn. In the button, the variable s is not localized in Manipulate. The frontend variable has the form \$CellContext`s, automatically linked to the kernel variable s, as seen on the button. The only noticable fact is that in the specification {{x,s},-1,1} only the value of s is used for the initialisation. It is not the variable Global`s.

Now we wrap this in another Manipulate, with s as control variable.

Manipulate[
  Manipulate[Row[{HoldForm[x], " = ", x}], {{x, s}, -1, 1}, 
     Button[Dynamic[Row[{HoldForm[s], " = ", Dynamic[s]}]], s = RandomReal[{-1, 1}]]],
  {s, -1, 1}]

The outer Manipulate localizes s to a frontend variable \$CellContext`s\$\$. In the inner Manipulate, it has to be displayed as argument of a kernel function, so a kernel variable FE`s$$mmm is created. With respect to the inner Manipulate the difference is that now the variable s is recognized by the outer Manipulate as a variable of the inner Manipulate, also in the specification {{x,s},-1,1}. Therefore, each time that we move the slider for s or press the button, the inner Manipulate is recomputed with initial value s for x. So everything is as it has to be.

There is not much documentation about frontend variables and how they are displayed in the boxes. The above outline is based on experiments and guessing. All comment, both corrections and extensions, are highly welcomed.