# Dynamic graphics are less responsive inside a Manipulate

I have some code which displays a GraphPlot and allows the user to click on any vertex in the graph to display some information about that vertex in a popup window. There is two-way highlighting between the popup and the vertex, so that if the user hovers over either the vertex or the popup window, both the vertex and the popup will be highlighted (the vertex by changing size and the popup by changing colour).

I do this by defining a VertexRenderingFunction for the graph plot which uses Module to create a unique symbol for each vertex. This symbol is set to True whenever the mouse is over the vertex or the popup window, and the dynamic highlighting is determined by the value of the symbol.

This works okay (though I'd be interested in better ways to accomplish the same effect). However, as soon as I put the GraphPlot inside a Manipulate, the highlighting becomes rather sluggish, even if the Manipulate is not changing the plot in any way. I don't understand why that happens.

Here's a minimal working example:

(* create some data *)
data = Thread[RandomChoice[Range[200], 350] -> RandomChoice[Range[200], 350]];

(* define the vertex rendering function *)
vrf := Module[{mo},
PopupWindow[
Dynamic[
mo = CurrentValue["MouseOver"];
{PointSize[Dynamic[If[mo, 0.03, 0.01]]], Point[#1]}],
Dynamic[
mo = CurrentValue["MouseOver"];
Panel["Popup Window Text", Background -> Dynamic[If[mo, Pink, White]]]]
]] &;

(* graph plot *)
gp = GraphPlot[data, VertexRenderingFunction -> vrf]


This works nicely. But if I do this:

Manipulate[gp, {x, 0, 10}, TrackedSymbols :> {x}]


it is much less responsive. Can anyone tell me why this happens, and more importantly how to prevent it?

I'm using version 8.0.4 by the way.

NB The first argument of the Manipulate in my example is deliberately independent of x. The slider control is nothing to do with the problem, it is just there to make a valid Manipulate expression. I have used TrackedSymbols :> {x} to make doubly certain that Manipulate isn't tracking any symbols inside the GraphPlot. In my real application the Manipulate does useful things, but I realised as I stripped the code down that merely being inside a Manipulate affects the responsiveness of the plot, even if the Manipulate doesn't change the plot in any way.

-
I can't reproduce the "horribly" part here yet –  Rojo Feb 17 '13 at 18:28
@Nasser, no I'm not moving the slider at all. The problem is with the highlighting of the dots as the mouse moves over them. I only put the control variable in there to make it a valid Manipulate expression. The TrackedSymbols was just to make doubly certain that Manipulate wasn't tracking any symbols inside the GraphPlot. I will edit the question to make this clearer. –  Simon Woods Feb 17 '13 at 20:38
With 8.0.4 I see substantial slowdown wrapping in Manipulate but barely any slowdown when wrapping in DynamicModule. I have a bias against Manipulate but in this case if you want to add other controls seems like DM is the way to go. (also tried Tooltip instead of popup windows and worked fine in DM). I've been attempting some larger scale dynamic data interfaces lately in order to mimic Tableau and finding that Mma probably hasn't been designed for interfaces with large numbers of tooltips, etc. –  Mike Honeychurch Feb 17 '13 at 21:45
If i change Module to DynamicModule and SetDelayed to Set in the definition of vrf, i get better responsiveness (ver 8.0.4 on Windows Vista). –  kguler Feb 17 '13 at 22:14
It remains unclear why different code is needed in Manipulate compared to DynamicModule to achieve comparable performance. Manipulate is a strange beast IMO. –  Mike Honeychurch Feb 18 '13 at 5:25

Putting the Dynamic wrapper of the point graphics outside the list seems to make a difference.

data = Thread[RandomChoice[Range[200], 350] -> RandomChoice[Range[200], 350]];

vrf = Module[{mo},
PopupWindow[Dynamic[mo = CurrentValue["MouseOver"];
Dynamic@{PointSize[If[mo, 0.03, 0.01]], Point[#1]},
TrackedSymbols -> {}], Dynamic[mo = CurrentValue["MouseOver"];
Panel["Popup Window Text", Background -> Dynamic[If[mo, Pink, White]]]]]] &;


I notice an improvement in both

gp = GraphPlot[data, VertexRenderingFunction -> vrf]


and especially in

Manipulate[gp, {x, 0, 10}]


The Manipulate is nearly as fast as the plain dynamic graphics.

I noticed that

Manipulate[gp]


does not behave as badly as the Manipulate with a slider, although perhaps slightly worse than just gp.

A wild guess is that Manipulate inserts an EventHandler in front of the output that causes extra checking. For instance, it might be that a "MouseOver" event is checked at all points instead of only the relevant one. I tried CurrentValue[foo++; "MouseOver"], which showed that it is updated only at one point, but that doesn't mean the front end doesn't do extra checking behind the scenes. [Update: This seems less likely after further investigation. See below.]

The wild guess does not really explain why the solution above works, though. It seems like familiarity with the internals workings of Manipulate will be needed to explain what the problem is.

Updated Edit: A better guess.

It seems that wrapping the visible object with Dynamic is to be preferred to wrapping Dynamic around code inside the object. For instance, as in the above or better yet as in @Rojo's comment,

Dynamic[PointSize[If...]]


is better than

PointSize[Dynamic[If...]]


Why might that be?

I stress might, because I don't know. The system is designed to dynamically update objects that appear in a visible cell in a notebook. Dependencies have to be parsed, and at a superficial glance, it seems to me it might be easier if the visible object is the dynamic object and not some internal part of it that in itself is not visible. For instance it would be better if Dynamic wrapped the PointSize directive and not the If statement inside PointSize. In any case, all the points are being updated when Dynamic is inside PointSize, as observed by @Rojo. Further there are four updates per mouse-over for each point (estimated), which is quite a lot.

Now the order of PointSize@Dynamic@... seems confusing. It seems to say PointSize, being outside, is not dynamic, only what is inside it. So what does the system do when the inside changes? The original code has

Dynamic[mo = CurrentValue["MouseOver"];
{PointSize[Dynamic[If[mo, 0.03, 0.01]]], Point[#1]}]


Notice that PointSize is itself inside another Dynamic. One ought to be able to remove the inside Dynamic.

An improvement

When the inside Dynamics are removed, you run into another problem already pointed out implicitly by @MikeHoneychurch that each of the two linked objects, the point and the popup, overwrites mo. So you need separate state variables. This led to the following:

vrf = Module[{mo1 = False, mo2 = False},
PopupWindow[Dynamic[mo1 = CurrentValue["MouseOver"];
{PointSize[If[mo1 || mo2, 0.03, 0.01]], Point[#1]}],
Dynamic[mo2 = CurrentValue["MouseOver"];
Panel["Popup Window Text", Background -> If[mo1 || mo2, Pink, White]]]]] &;


But further investigation reveals that there is still some inefficiency. There are two updates per mouse-over on a Point. A change CurrentValue["MouseOver"] causes an update to mo1; since PointSize depends on mo1, another update is incurred. This led me to the following changes.

A better improvement

Here is a more efficient implementation:

vrf = Module[{mo1 = False, mo2 = False},
PopupWindow[
Dynamic@{PointSize[If[(mo1 = CurrentValue["MouseOver"]) || mo2, 0.03, 0.01]], Point[#1]},
Panel["Popup Window Text",
Background -> Dynamic@If[mo1 || (mo2 = CurrentValue["MouseOver"]), Pink, White]]]] &;


It has one update per mouse-over. Putting the assignment inside If makes the expression for the point not depend on the value of mo1. Ditto for the expression for the Panel.

Why does Manipulate run so slowly? Hmm…

I'm beginning to agree with @MikeHoneychurch that Manipulate is a strange beast. I attempted to measure the CPU load by running my mouse quickly back and forth over closely connected points. My computer is a 2.5GHz 2-core Mac, and the monitor reports loads as percentages that can add up to 200% (100% per core). It's not an exact measurement, obviously, but it seems to reveal something about the behavior.

Using the original code, without Manipulate, the load is 80%-90%, about %50+ by the front end and 30%+ by the kernel, and the response time is ok but noticeably slower than any of the improvements. With Manipulate the load is about 100%+ when the graph is inside Manipulate, 40%+ by the front end, 60% by the kernel. Compared to the performance without Manipulate cited above, the decrease in the front end load is reflected in the slow performance. The substantial increase in the kernel load, about double, reflects a substantial amount of additional computation. However, the number of updates isn't different than the original dynamic graphics, four for each point (all points updated every mouse-over). By comparison, with any of the improvements I can get the front end up above 60% and the kernel remains below 2%, whether or not inside Manipulate, and the response is quite rapid.

I have not been able to think of a reason for the increased demand on the kernel inside.

-
or you could just use Mouseover: vrf2 := PopupWindow[ Mouseover[{PointSize[0.01], Point[#1]}, {PointSize[0.03], Point[#1]}], Mouseover[Panel["Popup Window Text", Background -> White], Panel["Popup Window Text", Background -> Pink]]] &; or without DynamicModuel: vrf1 := PopupWindow[ Dynamic[{PointSize[If[CurrentValue["MouseOver"], 0.03, 0.01]], Point[#1]} ], Dynamic[ Panel["Popup Window Text", Background -> If[CurrentValue["MouseOver"], Pink, White]] ]] &; –  halmir Feb 18 '13 at 18:30
Thanks Michael, moving the Dynamic outside PointSize was Rojo's suggestion too, and it seems to be the key to getting decent performance. –  Simon Woods Feb 18 '13 at 20:25
@halmir, I need the highlighting to happen on both the vertex and the popup when the mouse is over either of them. Hence the need for a symbol to hold the current mouseover state. –  Simon Woods Feb 18 '13 at 20:29
Should this be a community wiki too? Or part of the other one? I'm afraid I don't completely understand the etiquette or purpose of them. –  Michael E2 Feb 19 '13 at 16:01
@MichaelE2, in this case I think Mike made the answer a community wiki simply because it was a combined effort to find a solution. I think your answer adds some interesting ideas about the cause of the problem, so I would leave it as it is. –  Simon Woods Feb 19 '13 at 20:44

As per comment thread the thing that produces a more responsive Manipulate is to use Dynamic[Point[If[...]]] rather than Point[Dynamic[If[...]]]. Using two mouse over variables enables you to mouse over the popup and change the size of the point.

(*create some data*)
RandomChoice[Range[200], 350] -> RandomChoice[Range[200], 350]];

(*define the vertex rendering function*)

vrf = DynamicModule[{mo1 = False, mo2 = False},
PopupWindow[
Dynamic[mo1 =
CurrentValue["MouseOver"]; {Dynamic@
PointSize[If[mo1 || mo2, 0.03, 0.01]], Point[#1]}],
Dynamic[mo2 = CurrentValue["MouseOver"];
Panel["Popup Window Text",
Background -> Dynamic@If[mo2, Pink, White]]]]] &;

(*graph plot*)gp = GraphPlot[data, VertexRenderingFunction -> vrf]


and the Manipulate:

Manipulate[gp, {x, 0, 10}, TrackedSymbols :> {x}]

-