Unexpected Setter Bar Behavior when Manipulate Nested in Dynamic Module

Question

Consider this code:

DynamicModule[{x, y},
x[t_] := t^2 + 5;
y[t_] := t^3 + 5;

Manipulate[Plot[fcn[t], {t, -5, 5}],
{fcn, {x -> "Square", y -> "Cubic"}}]]

Why does the SetterBar created by Manipulate not highlight properly when either option is selected?

The same problem exists for other user interface elements, like a RadioButtonBar:

DynamicModule[{x, y},
x[t_] := t^2 + 5;
y[t_] := t^3 + 5;

Manipulate[Plot[fcn[t], {t, -5, 5}],
{{fcn, x, "Graph Type"}, {x -> "Square", y -> "Cubic"}, RadioButtonBar}]]

However, if you remove the DynamicModule, everything works as expected:

x[t_] := t^2 + 5;
y[t_] := t^3 + 5;

Manipulate[Plot[fcn[t], {t, -5, 5}],
{{fcn, x, "Graph Type"}, {x -> "Square", y -> "Cubic"}}]

Also if you were to replace DynamicModule with Block or Module, everything works as expected:

Module[{x, y},
x[t_] := t^2 + 5;
y[t_] := t^3 + 5;

Manipulate[Plot[fcn[t], {t, -5, 5}],
{{fcn, x, "Graph Type"}, {x -> "Square", y -> "Cubic"}}]]

Can anyone explain why this is, and how to fix it?

Answer 1

I hope you like explanations. I should point out that I will assume the reader is familiar with the element of Mathematica's dynamic updating system, as described in the introductory tutorials [1], [2],, but some of the issues touch on more advanced details found in [3], [4].

What's going on

Manipulate does many things automatically, which can lead to issues when mixing it up too much with Dynamic and DynamicModule. One of the issues is that Manipulate goes through the code it builds and any argument to a function in the control/variable specifications that has a symbol in the FE` context (seems to) get wrapped in Dynamic. If the function does not handle that properly, then the code will break.

Another issue is that DynamicModule (and by implication Manipulate) does not create new Unique'd symbols when it is evaluated in the Kernel. They get created when the DynamicModule is instantiated in the Front End (that is, when the Front End displays the DynamicModule output. If x is a variable xin a DynamicModule, the Kernel substitutes the symbol x$$ in the DynamicModule output that is returned to the Front End. That is the end of the first step.

The next step happens in the Front End, when the Front End displays the DynamicModule output. In the Front End, x$$ is converted to a symbol in the context FE` of the form FE`x$$nnn, where nnn are numerals appended to the name to make it unique, when it is to be dynamically updated. Basically a symbol is to be dynamically updated when appears inside a Dynamic; but it might also be in a function that ultimately returns a Dynamic expression. As the Front End instantiates the DynamicModule, it send this converted symbol to the Kernel for initialization together with the rest of the DynamicModule code. In the example below, it turn out the symbols for x and y were initiated with Unset[FE`x$$14`] and FE`y$$14 = 2 are sent to the kernel. ("Show Expression" output shown below.)

DynamicModule[{x, y = 2}, Dynamic@{x, y}]
(* {FE`x$$14, 2} *)

 Cell[BoxData[ 
  DynamicModuleBox[{$CellContext`x$$, $CellContext`y$$ = 2},

DynamicBox[ToBoxes[{$CellContext`x$$, $CellContext`y$$}, StandardForm], ImageSizeCache->{172., {3.608642578125, 14.}}], DynamicModuleValues:>{}]], "Output", CellChangeTimes->{3.584173863470374*^9, 3.584174940037945*^9}]

In the next example below we can see that a variables x etc. are converted from a $$ symbol to an FE` symbol when it appears inside Dynamic. These are stored in the output Cell as $CellContext`x$$, which the Front End converts when it displays the dynamic module. The symbols not wrapped in a Dynamic are stored just as x$$ without the $CellContext` context; and these are not converted. (This is not the whole story on when they are converted, by the way.) In fact these were evaluated in the Kernel when the DynamicModule was first evaluated. This is why a 2 was returned instead of y$$ in the second sublist. When Dynamic@{x, y} is evaluated in the Kernel, it is basically passed right back to the Front End (actually in a DynamicBox with x$$, y$$), which will it evaluate when it gets ready to display the expression in the output. At that point, the FE` symbols are substituted.

DynamicModule[{x, y}, y = 2; {Dynamic@{x, y}, {x, y}, Hold[y], Dynamic@Hold[y]}]

So we see that the variable x in the code is translated to two distinct variables in the output.

The reason this is an issue for the OP's code has to do with the values that are the basis for the SetterBar. Are they going to be x$$, y$$ or FE`x$$nnn, FE`y$$nnn? We've seen that both are possible. If the first, then the SetterBar won't highlight, even though it seems to set fcn properly, because when fcn = x is executed inside the Dynamic[fcn] that is passed to the SetterBar, fcn will be set to FE`x$$nnn after conversion by the Front End. This value will not match the x$$ passed to SetterBar, and the SetterBar will not be highlighted. This is the problem with the nonworking Manipulates in the question. Fix this and the Manipulate should work (see "Fixes" below).

A rather minor issue is that Manipulate does not initialize a variable the way one might expect with the declaration

{var, {d1 -> "label 1", d2 -> "label 2", ...}}

Instead of var = d1, we get var = d1 -> "label 1". You have to explicitly initialize var with

{{var, d1}, {d1 -> "label 1", d2 -> "label 2", ...}}

I say it's a minor issue because if d1 is a Manipulate variable, then there is still another issue having to do with how Manipulate builds the code for the SetterBar.

Fixes

In addition to the other fine answers, here are some other options. There are a few I wouldn't recommend (and so indicated), but are included for illustration purposes. The other methods I've used in practice, depending on what was convenient in a given situation.

Dynamic. This fix occurred to me while thinking through the reasoning above. I would not recommend it in practice, although its working correctly is consistent with the explanation.

However, if you try to initialize fcn with {{fcn, x}, ...} it gets initialized to y (the last element in the list). I do not know why. In the current form fcn will be initialized to {x -> "Square", ...}; you could uncomment the If statement to make a one-time initialization.

DynamicModule[{x, y},
 x[t_] := t^2 + 5;
 y[t_] := t^3 + 5;
 Manipulate[
  (*If[MatchQ[fcn, _List], fcn = x];*)
  Plot[fcn[t], {t, -5, 5}],
  {fcn, {x -> "Square", y -> "Cubic"}, Dynamic @ SetterBar[#1, First@#2] &}]]

Pure Functions.

Manipulate[
 Plot[fcn[t], {t, -5, 5}],
 {{fcn, #^2 + 5 &}, {#^2 + 5 & -> "Square", #^3 + 5 & -> "Cubic"}}]

Use expressions (three ways)

Global variables

Manipulate[
 Plot[fcn, {t, -5, 5}],
 {{fcn, t^2 + 5}, {t^2 + 5 -> "Square", t^3 + 5 -> "Cubic"}}]

Local Kernel variables

Module[{t},
 Manipulate[
  Plot[fcn, {t, -5, 5}],
  {{fcn, t^2 + 5}, {t^2 + 5 -> "Square", t^3 + 5 -> "Cubic"}}]
 ]

Local Manipulate variables. This way SetterBar must be explicitly specified; otherwise you get a slider for fcn. This inexplicable behavior has to do with what's going on under the hood of Manipulate, and I don't know what's going on under the hood of Manipulate. It hinges on t being one of the Manipulate variables, but I cannot explain why or how. It suggests that this may not be the most reliable solution, but it works here.

Manipulate[
 Plot[fcn, {t, -5, 5}],
 {{fcn, t^2 + 5}, {t^2 + 5 -> "Square", t^3 + 5 -> "Cubic"}, SetterBar},
 {{t, t}, None}]

Localize function symbols in Module, which you did in one of your examples. I suggest initializing them in the Manipulate.

Module[{x, y},
 Manipulate[
  Plot[fcn[t], {t, -5, 5}],
  {{fcn, x}, {x -> "Square", y -> "Cubic"}},
  Initialization :> (
    x[t_] := t^2 + 5;
    y[t_] := t^3 + 5)]
 ]

Rules

Manipulate[
 Plot[Evaluate[name[t] /. nameToFcn], {t, -5, 5}],
 {{name, "x", "function"}, {"x" -> "Square", "y" -> "Cubic"}},
 {{x, x}, ControlType -> None},
 {{y, y}, ControlType -> None},
 {nameToFcn, ControlType -> None},
 Initialization :> (
   x[t_] := t^2 + 5;
   y[t_] := t^3 + 5;
   nameToFcn = {"x" -> x, "y" -> y})]

Play the Front End for a sucker: Create the symbol you need. Probably not a good solution, but interesting. This avoids having a DynamicModule variable in the data {"x$$" -> "Square", "y$$" -> "Cubic"} that is passed to SetterBar, so Manipulate passes it as is. (The body of a Manipulate is wrapped in Dynamic automatically.)

DynamicModule[{x, y},
 x[t_] := t^2 + 5;
 y[t_] := t^3 + 5;
 Manipulate[
  Plot[Evaluate[Symbol[fcn][t]], {t, -5, 5}],
  {{fcn, "x$$"}, {"x$$" -> "Square", "y$$" -> "Cubic"}}]]

DynamicModule with OwnValues. (Edit per request in comment.)

Instead of UpValues, x[t_] :=.., if we use OwnValues, x = .., together with our own SetterBar function, we can make DynamicModule work with Manipulate in this case:

DynamicModule[{x, y},
 Manipulate[
  Plot[Evaluate[fcn[t]], {t, -5, 5}],
  {{fcn, x}, 
   SetterBar[Dynamic@fcn, {x -> "Square", y -> "Cubic"}] &},
  Initialization :> (
    x := t \[Function] t^2 + 5;
    y := t \[Function] t^3 + 5)]
 ]

But it's simpler to Manipulate as our DynamicModule -- it's more or less the same, but without the nesting which makes things less complicated (and more predictable).

Manipulate[
 Plot[Evaluate[fcn[t]], {t, -5, 5}],
 {{fcn, x}, SetterBar[Dynamic@fcn, {x -> "Square", y -> "Cubic"}] &},
 {x, ControlType -> None}, {y, ControlType -> None},
 Initialization :> (
   x = t \[Function] t^2 + 5;
   y = t \[Function] t^3 + 5)]

The issue as explained above really lies in passing the right values to SetterBar in the list {value -> label, ..}. Because SetterBar is inside a Function (SetterBar[..] &), the SetterBar is not evaluated until the dynamic module is instantiated by the Front End. So the variables x and y in the second argument will be evaluated at that time, when they will be converted to the form FE`x$nnn.

Seeing what SetterBar is getting

I wrote a little helper control that stored some of the arguments to help me see how Manipulate handled things. A declaration of the form

{var, list, function}

results in function[Dynamic[var], {list}] or function[Dynamic[var], {Dynamic[list]}], depending on whether Manipulate thinks the extra Dynamic is helpful. This seems to happen when list contains Manipulate variables. You have to strip the Dynamic off, though, or SetterBar won't recognize it as a valid argument. But that's part of the problem. The values are going to be of the form x$$, not FE`x$$nnn, and the SetterBar won't highlight. Presumably Manipulate put Dynamic around them to ensure they be converted properly.

Clear[mySB];
mySB[Dynamic[x_], {Dynamic[vals_]}] :=
  (setterx = Hold[x]; settervals = Dynamic[vals]; 
   SetterBar[Dynamic[x, (updatex = Hold[x]; x = #) &], vals]);
mySB[Dynamic[x_], {vals_}] := (setterx = Hold[x]; settervals = vals; 
   RadioButtonBar[Dynamic[x, (updatex = Hold[x]; x = #) &], vals]);

Interestingly, one can see that while fcn$$ is passed as an argument to the control, it is FE`fcn$$nnnn that is updated dynamically. (Click on a control to cause updatex to be set.)

Manipulate[
 fcn,
 {{fcn, x}, {x -> "Square", y -> "Cubic"}, mySB[##] &},
 {{x, x}, ControlType -> None},
 {{y, y}, ControlType -> None},
 TrackedSymbols :> {fcn}
 ]
(* output omitted *)

settervals // FullForm
setterx
updatex
(* Dynamic[List[Rule[x$$, "Square"], Rule[y$$, "Cubic"]]] *)
(* Hold[fcn$$] *)
(* Hold[FE`fcn$$2501] *)

When the x, y are put in an outer DynamicModule, Manipulate does not wrap them in Dynamic.

DynamicModule[{x, y},
 Manipulate[
  fcn,
  {{fcn, x}, {x -> "Square", y -> "Cubic"}, (mySB[##]) &},
  TrackedSymbols :> {fcn}
  ]
 ]
(* output omitted *)

settervals // FullForm
setterx
updatex
(* List[Rule[x$$, "Square"], Rule[y$$, "Cubic"]] *)
(* Hold[fcn$$] *)    
(* Hold[FE`fcn$$2513] *)

Answer 2

Module variables should *never* appear inside Dynamics or Manipulates internal to that Module. [John Fultz, 2011]

Since Manipulate itself is DynamicModule Hence you effectively have this:

DynamicModule[{x, y}, x[t_] := t^2 + 5;
 y[t_] := t^3 + 5;
 Manipulate[
  Plot[fcn[t], {t, -5, 5}], {fcn, {x -> "Square", y -> "Cubic"}}]]

turned to

 DynamicModule[{x, y}, x[t_] := t^2 + 5;
  y[t_] := t^3 + 5;
  DynamicModule[{fcn = FE`x$$12 -> "Square"}, Plot[fcn[t], {t, -5, 5}]]]

You can see how Manipulate became a DynamicModule by hitting the small + on the top corner of the Manipulate and clicking Paste snapshot.

So, what does this mean, you ask? Well, you hit one of the bad rules of thumbs in Dynamics, you had a Module variable x appears inside a Dynamic internal to that Module, i.e. you have this pattern:

 Module[{a}, Dynamic[a]] 

Mixing Dynamics inside Dynamics is a tricky thing. I just use the famous John Fultz rules on this from his post on sept 2011, and I never have problems after that (well, almost never :). Here are the rules

I could explain in some considerable detail exactly what's going on here (and it would be considerable), but it might be more useful and straightforward to give you a very simple rule...

Module variables should never appear inside Dynamics or Manipulates internal to that Module.

To be clear with some examples (all using Dynamic, but they could equally well use Manipulate, which is implemented using Dynamic)...

(* OK *) Dynamic[Module[{a}, a]]

(* OK ) Module[{a}, ( expression involving a*); Dynamic[(* expression not involving a *)]

(* BAD *) Module[{a}, Dynamic[a]]

In fact, this rule is so straightforward and universal, there's no reason that we shouldn't have some advisory syntax highlighting which points out the badness...I intend to do this for a future release.

Answer 3

Nasser gives a learned discussion of why, but he doesn't tell how to fix it. It's easy:

Manipulate[
  Plot[fcn[t], {t, -5, 5}], 
  {{fcn, x, "function"}, {x -> "Quadratic", y -> "Cubic"}},
  Initialization :> (
    x[t_] := t^2 + 5;
    y[t_] := t^3 + 5)]

Note that I made a small modification to your setter bar specification. In this form the Manipulate will start up with "Quadratic" choice selected.