# Automate mouse clicks with Mathematica

I need to automate mouse clicks with Mathematica. However, a similiar question was closed on this forum, without any answer.

Consider the following steps:

 Dynamic[MousePosition[]]


In this first step I dynamically show the mouse position.

 DynamicModule[{col = Green}, EventHandler[ Style["text", FontColor ->
Dynamic[col]], {"MouseClicked" :> (col = col /. {Red -> Green, Green -> Red})}]]


In this second step I create a "clickable" text: when you click it, it changes its color between green and red.

My objective here is twofold:

1) automatically move the cursor position - this has already been answered here. So, following the original answer I can write

Needs["JLink"];
ReinstallJava[];
robotclass = JavaNew["java.awt.Robot"];
robotclass@mouseMove[90, 196];


2) automatically click on the clickable text (and, of course, I want to see its color changing). Here is the code I'm trying to use:

robotclass@mousePress[InputEvent.BUTTON1_MASK];


Although I believe I'm using the correct functions (which belong to the Java Robot Class), I'm getting the following error message:

Java::argx1: Method named mouseRelease defined in class java.awt.Robot was called with an incorrect number or type of arguments. The argument was InputEvent.BUTTON1_MASK.

Can anybody give me a hint to solve this problem?

-

You can use the raw integer value 16, but the correct J/Link syntax for the symbolic name is InputEventBUTTON1UMASK, so your code would look like this:

(* Call LoadJavaClass when needing to refer to static members. *)

robotclass@mousePress[InputEventBUTTON1UMASK];


Enter JLink/tutorial/CallingJavaFromMathematica#15615 into the Documentation Center to see the section of the J/Link docs that talks about how to construct this name. BUTTON1_MASK is a static member of the InputEvent class, hence the , and the _ character needs to be converted into a U to make it a legal Mathematica symbol name.

-
It also works! +1 –  Rod Aug 19 at 10:27
hmm wonder why this syntax didn't occur to me. @RodLm feel free to change the accepted question –  amr Aug 19 at 14:37
InputEvent.BUTTON1_MASK is an attempt to get the dot product of InputEvent and BUTTON1_MASK, neither of which are matrices. You're using Java syntax in a Mathematica program.
I don't know what the idiomatic JLink approach is, but if you look here you can see that InputEvent.BUTTON1_MASK has the integer value 16 (the BUTTON1_MASK is just a variable for programmer convenience -- easier to deal with than an arbitrary integer).
I've tried other sintaxes as well... for instance, instead of InputEvent.BUTTON1_MASK I've also tried InputEvent[BUTTON1_MASK], mouseRelease["InputEvent.BUTTON1_MASK"], 'mouseRelease[InputEvent,BUTTON1_MASK]' and so on... but nothing seems to work... By the way, you could have put a comment instead of an answer... anyway, thank you. –  Rod Aug 19 at 3:42
@RodLm for some reason i had assumed you were familiar with Java. to be clearer: the mousePress function expects an integer. mousePress(InputEvent.BUTTON1_MASK) works in Java because InputEvent.BUTTON1_MASK is an integer in Java. what you do with this knowledge is something i leave to you :) –  amr Aug 19 at 3:55
+1! It works... You only need to use robotclass@mousePress[16]` and it works perfectly. Thanks! –  Rod Aug 19 at 3:58