enter image description here

I can select all the boxes in a StandardForm rendered expression, like I've selected the left parenthesis in the picture above. Is it possible to get the positions and dimensions of all of these selectable boxes? Clearly the Mathematica rendering engine has this information, it's just a question of getting access to it somehow. Might J/Link provide this access?

  • $\begingroup$ I notice that you Accepted my answer. Were you able to complete the idea and produce a working implementation? If so would you consider posting it? If not you might hold off on the Accept in case someone else has a better idea. Thank you in any case. $\endgroup$
    – Mr.Wizard
    May 15 '15 at 1:34

This is not yet a complete answer but I think it may set you on a viable path. We can style each element using StyleBox as follows:

boxes = RowBox[{"(", 
     RowBox[{RowBox[{"(", RowBox[{"a", "+", "b"}], ")"}], SuperscriptBox["c", "d"]}], 
     SqrtBox["e"]], ")"}];

colors = ColorData[54, "ColorList"]

boxesNew =
  Module[{i = 1},
    boxes /. s_String :>
      (StyleBox[s, FontColor -> #, Background -> #] & @ colors[[i++]])

The result:

boxesNew // DisplayForm

enter image description here

The object selected, revealing its parts:

enter image description here

We can create an Image with Rasterize:

img = Rasterize[boxesNew // DisplayForm, "Image"];

This image could then be processed to identify each area of solid color and these mapped back to the color list in colors. (The colors chosen are only for human readability; sequential RGB values might be used for many segments.)


Details depend of your specific needs but it seems that information you desire is at least partially accesible via FrontEnd`UndocumentedGetSelectionPacket.

    RowBox[{RowBox[{"(", RowBox[{"a", "+", "b"}], ")"}], 
      SuperscriptBox["c", "d"]}], SqrtBox["e"]], ")"}
] // RawBoxes

 FrontEndExecute @ FrontEnd`UndocumentedGetSelectionPacket[EvaluationNotebook[]], 
 UpdateInterval -> 1

enter image description here

VirtualSelectionRectangle seems to give {xmin, xmax}, {ymin, ymax} where x are pixel columns counted from left edge of the notebook and y coordinate determines pixel row counting from top edge (NOT from the beginning of the notebook).

  • $\begingroup$ This is really neat! But how would one programmatically apply this to every selectable part? This might make for a good self-Q&A that is more directly answered by UndocumentedGetSelectionPacket. $\endgroup$
    – Mr.Wizard
    Aug 30 '16 at 0:05
  • $\begingroup$ @Mr.Wizard I suppose a loop with SelectionMove should do that. Not without pain but seems possible. $\endgroup$
    – Kuba
    Aug 30 '16 at 5:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.