13
$\begingroup$

I notice that the built-in "Coordinates Tool" has a very efficient zoom tooltip displaying an enlarged portion of the image along with coordinates and row/column indices:

screenshot

How to get the source code of this tool? Or how can we implement such an efficient tooltip ourselves (I want to replicate all the tooltip functionality)?

$\endgroup$
10
  • $\begingroup$ How to get the source code of this tool? get a job at WRI and then you can look at all the source code :) sorry, could not help it. I wonder if it is using Dynamics somehow? $\endgroup$
    – Nasser
    Commented Sep 17, 2022 at 8:33
  • $\begingroup$ @Nasser Often there is an undocumented way to get the source. For example, via FrontEndResource or some other trick. So I don't consider the question unanswerable. $\endgroup$ Commented Sep 17, 2022 at 10:02
  • 3
    $\begingroup$ It looks like the tool is implemented entirely in the frontend: If you have an image selected while evaluating FrontEndExecute[FrontEnd`Select2DTool["GetPixelPointMarkers"]], you will get the tooltip. However, the "select 2D tool" name somehow implies that the frontend knows how to select the tool, with no kernel involvement (you can also see by looking at the CPU usage of the various threads that the kernel doesn't seem to do much, if anything). The string "GetPixelPointMarkers" also only appears a single time in the entire installation directory $\endgroup$
    – Lukas Lang
    Commented Sep 17, 2022 at 11:27
  • 1
    $\begingroup$ @AlexeyPopkov It's in FileNameJoin@{$InstallationDirectory, "SystemFiles", "FrontEnd", "SystemResources", "AttedImage2D.nb"} $\endgroup$
    – Lukas Lang
    Commented Sep 17, 2022 at 12:49
  • 1
    $\begingroup$ @LukasLang Thanks. The list of indices can be obtained with MathLink`CallFrontEnd[ FrontEnd`Value[FEPrivate`GetlImageMarkersIndicesList[NotebookSelection[]]]]. $\endgroup$ Commented Sep 17, 2022 at 12:54

2 Answers 2

13
$\begingroup$

Here's a proof-of-concept demonstration of how to build your own tooltip with reasonable performance:

tooltip[img_] :=
 EventHandler[
  MouseAppearance[
   img,
   Dynamic@Graphics[
     {
      Red,
      Line@{{{0.05, 0.9}, {0.05, 1}}, {{0, 0.95}, {0.1, 0.95}}},
      Inset[
       Graphics[
        {
         Inset[img,
          {1, 1},
          {
           FEPrivate`Ceiling@FEPrivate`Part[FrontEnd`MousePosition["Graphics"], 1],
           FEPrivate`Ceiling@FEPrivate`Part[FrontEnd`MousePosition["Graphics"], 2]
           },
          ImageDimensions@img
          ],
         EdgeForm@Red,
         FaceForm@None,
         Rectangle[{0, 0}, {1, 1}]
         },
        PlotRange -> {{-4, 5}, {-4, 5}},
        Frame -> True, FrameTicks -> None, FrameStyle -> Thick,
        PlotRangeClipping -> True
        ],
       {0.1, 0.9},
       Scaled@{0, 1},
       Scaled@{0.9, 0.9}
       ]
      },
     ImageSize -> 125,
     PlotRange -> {{0, 1}, {0, 1}}
     ],
   {0.05, 0.95}
   ],
  {
   "MouseClicked" :> Print@MousePosition@"Graphics"
   }
  ]

tooltip@ExampleData[{"TestImage", "House"}]

enter image description here

Notes:

  • The tooltip is shown by setting the mouse appearance via MouseAppearance. This allows the tooltip to smoothly follow the cursor position. The red cross is aligned such that the alignment point of the cursor (set to {0.05,0.95}) is at its center
  • The zoomed image displayed is achieved by a series of nested Insets. This allows us to crop the image and move it around without actually doing any image processing, we are simply changing the shown portion of the image.
  • The part with FEPrivate`Ceiling@... is effectively computing the inset position in the front end, and rounding it to the appropriate pixel value (to get the pixel snapping behavior).
  • The coordinates of the innermost image graphics are both set to reflect the ImageDimensions, so 1 unit is 1 pixel. This makes many things a lot easier, especially the control of how many pixels are shown: This is essentially driven by the PlotRange->{{-4,5},{-4,5}} setting.
$\endgroup$
7
  • $\begingroup$ Great! One problem: when I add "MouseClicked":>Print[MousePosition["Graphics"]], it prints the coordinates, but the tooltip disappears. How can it be avoided? $\endgroup$ Commented Sep 17, 2022 at 12:45
  • 1
    $\begingroup$ @AlexeyPopkov Please see the update - I just realized that you can set the cursor via MouseAppearance in this case, which makes everything more robust, and it also fixes the issue you mentioned $\endgroup$
    – Lukas Lang
    Commented Sep 17, 2022 at 12:54
  • $\begingroup$ Another problem: if the image contains the ImageResolution option, the tooltip breaks. Even with ImageResolution -> Automatic, the tooltip is blurred. Addition of BaseStyle->{Antialiasing->False} doesn't help. $\endgroup$ Commented Sep 17, 2022 at 15:40
  • $\begingroup$ As a workaround setting ImageResolution->72 works. $\endgroup$ Commented Sep 17, 2022 at 16:26
  • $\begingroup$ It is always a nice surprise to see new ways to create robust and performant applets in Mathematica. But I really did wish all these features were better documented and explained. $\endgroup$
    – QuantumDot
    Commented Sep 18, 2022 at 23:56
10
$\begingroup$

Answering the first part of my question (thanks to Lukas Lang for pointing me in the right direction!).

The source code for the Image Assistant Toolbar that appears when you select Image in the FrontEnd is in the file

FileNameJoin@{$InstallationDirectory, "SystemFiles", "FrontEnd", "SystemResources", 
  "AttachedImage2D.nb"}

In particular, the zooming tooltip can be invoked by evaluating

FrontEndExecute[FrontEnd`Select2DTool["DisplayImageTooltip"]]

when an Image is selected. It appears to be implemented at a low level in the FrontEnd, so the source code is not available.

The option CurrentValue[$FrontEnd, "DisplayImagePixels"] determines whether to show the zoomed portion of the image in the tooltip. The default value for this option is Automatic, what is equivalent to "ExploreView". With the value "TooltipInfo" the tooltip will be shown without the zoomed image. For displaying the zoomed image, we need to set "DetailExploreView":

CurrentValue[$FrontEnd, "DisplayImagePixels"] = "DetailExploreView";
SelectionMove[PreviousCell[], All, CellContents]
FrontEndExecute[FrontEnd`Select2DTool["DisplayImageTooltip"]]

screenshot

The pixel size of the zoomed image can be controlled via CurrentValue[$FrontEnd, "RasterExploreViewRange"].

Evaluating

FrontEndExecute[FrontEnd`Select2DTool["GetPixelPointMarkers"]]

activates also the point marker creation functionality. After creating the markers, the list of their row/column indices can be obtained with

MathLink`CallFrontEnd[FrontEnd`Value[FEPrivate`GetlImageMarkersIndicesList[NotebookSelection[]]]]

The makers can be removed with

FrontEndTokenExecute["ClearPixelPointMarkers"]

With the above, we can develop the following simple pixel picker interface:

PixelPickerInterface[i_Image] := 
  With[{id = CreateUUID[], id2 = CreateUUID[], 
    is = {{1, #}, {1, #}} &[101./CurrentValue["WindowResolution"]*72]},
   With[{init := (CurrentValue[$FrontEnd, "DisplayImagePixels"] = "DetailExploreView";
       SelectionMove[First@Cells[CellTags -> id], All, CellContents, 
        AutoScroll -> False];
       SelectionMove[EvaluationNotebook[], All, Graphics, AutoScroll -> False];
       SetOptions[
        First@Cells[NotebookSelection[], CellTags -> "AttachedImage2D", 
          AttachedCell -> True], CellSize -> {0, 0}];
       FrontEndExecute[FrontEnd`Select2DTool["GetPixelPointMarkers"]])},
    CellPrint[
     ExpressionCell[i, "Output", CellTags -> id, CellFrameLabels -> {{None, None}, {None,
         ToBoxes@Pane[Row[{Spacer[40],
             Button["Start Image Assistant Interface", init],
             Button["Print Markers Indices",
              NotebookDelete[Cells[CellTags -> id2]]; init;
              With[{inds = 
                 MathLink`CallFrontEnd[
                  FrontEnd`Value[
                   FEPrivate`GetlImageMarkersIndicesList[NotebookSelection[]]]]},
               CellPrint@Table[
                 ExpressionCell[
                  Row[Prepend[
                    Table[Image[
                    ImageTake[i, {-sh, sh} + ind[[1, 1]], {-sh, sh} + ind[[2, 2]]], 
                    ImageSize -> is], {sh, {0, 1, 4, 10, 50}}], 
                    Labeled[{ind[[1, 1]], ind[[2, 2]]}, "row,col"]], "->"], "Echo", 
                  CellTags -> id2], {ind, inds}]]],
             Button["Clear Markers",
              init; FrontEndTokenExecute["ClearPixelPointMarkers"]],
             Slider[Dynamic[CurrentValue[$FrontEnd, "RasterExploreViewRange"]], {39, 
               3, -1}, ImageSize -> Small],
             Dynamic[CurrentValue[$FrontEnd, "RasterExploreViewRange"]]
             }], Full, Alignment -> Left]}}]]
    ]];


PixelPickerInterface[ExampleData[{"TestImage", "House"}]]

screesnhot

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.