Automate mouse clicks with Mathematica

Question

I need to automate mouse clicks with Mathematica. However, a similiar question was closed on this forum, without any answer.

Consider the following steps:

 Dynamic[MousePosition[]]

In this first step I dynamically show the mouse position.

 DynamicModule[{col = Green}, EventHandler[ Style["text", FontColor -> 
 Dynamic[col]], {"MouseClicked" :> (col = col /. {Red -> Green, Green -> Red})}]]

In this second step I create a "clickable" text: when you click it, it changes its color between green and red.

My objective here is twofold:

1) automatically move the cursor position - this has already been answered here. So, following the original answer I can write

Needs["JLink`"];
ReinstallJava[];
robotclass = JavaNew["java.awt.Robot"];
robotclass@mouseMove[90, 196];

2) automatically click on the clickable text (and, of course, I want to see its color changing). Here is the code I'm trying to use:

robotclass@mousePress[InputEvent.BUTTON1_MASK];
robotclass@mouseRelease[InputEvent.BUTTON1_MASK];

Although I believe I'm using the correct functions (which belong to the Java Robot Class), I'm getting the following error message:

Java::argx1: Method named mouseRelease defined in class java.awt.Robot was called with an incorrect number or type of arguments. The argument was InputEvent.BUTTON1_MASK.

Can anybody give me a hint to solve this problem?

Answer 1

You can use the raw integer value 16, but the correct J/Link syntax for the symbolic name is InputEvent`BUTTON1UMASK, so your code would look like this:

(* Call LoadJavaClass when needing to refer to static members. *)
LoadJavaClass["java.awt.event.InputEvent"];

robotclass@mousePress[InputEvent`BUTTON1UMASK];
robotclass@mouseRelease[InputEvent`BUTTON1UMASK];

Enter JLink/tutorial/CallingJavaFromMathematica#15615 into the Documentation Center to see the section of the J/Link docs that talks about how to construct this name. BUTTON1_MASK is a static member of the InputEvent class, hence the `, and the _ character needs to be converted into a U to make it a legal Mathematica symbol name.

Answer 2

InputEvent.BUTTON1_MASK is an attempt to get the dot product of InputEvent and BUTTON1_MASK, neither of which are matrices. You're using Java syntax in a Mathematica program.

I don't know what the idiomatic JLink approach is, but if you look here you can see that InputEvent.BUTTON1_MASK has the integer value 16 (the BUTTON1_MASK is just a variable for programmer convenience -- easier to deal with than an arbitrary integer).