Can WebExcute click on a button on a webpage that has been created with Javascript?

Question

In an effort to explore the new WebExecute capabilities in Version 12 I have created a simple algorithm to download some data that are stored on an external server and displayed on a webpage. I am using the datatable from https://trumpgolfcount.com/displayoutings as a test, since the authors have no problem of making the data fully available to anyone either in .XLS or JSON format. Thus, allowing me to check if my algorithm is working as intended.

My WebExecute Script correctly loads the first page of this data and automatically generates a Mathematica Dataset from the data generated by the Javascript program displaying the data:

session = StartWebSession["Chrome"];
WebExecute[session, "OpenWebPage" ->  http://trumpgolfoutings/displayoutings"];
rowheaders = WebExecute[session, "LocateElements" -> "Tag" -> "th"];
rowrecords = WebExecute[session, "LocateElements" -> "Tag" -> "td"];
headers = WebExecute["ElementText" -> rowheaders];
numberOfHeaders = Length[headers];
golfrecords = WebExecute["ElementText" -> rowrecords];    
newgolfrecords = Partition[golfrecords, numberOfHeaders];
dataset = Dataset[Association /@ (MapThread[Rule, {headers, #}] & /@ newgolfrecords)];
DeleteObject[session];

All this works fine to upload part of the original table into dataset. However, the data are provided 100 records at a time, with buttons to move from one page of data to another also generated in Javascript and not as hyperlinks. Because subsequent data are also generated from the encoded Javascript the dataset is incomplete. Since there are now over 200 records, my algorithm only creates a dataset only for the first 100 records.

Is there a way using WebExecute (or an alternative route) to effect the pressing of these script generated buttons so that the above code could be repeated for each page containing additional records, all while under control of the WebExecute session?

Although the documentation for WebExectue discusses handling input to input boxes, how to perform an action such as a click doesn't seem to be available except in the contexts of executing hyperlinks via the hyperlink anchor element.

Answer 1

I would probably do it like this.

1. Start a new session:

browser = StartWebSession[]

2. Open the page in question:

WebExecute[browser, "OpenPage" -> "https://trumpgolfcount.com/displayoutings"]

3. Get the innerText of the table:

tableText = WebExecute[browser, "JavascriptExecute" -> 
    "return document.getElementById('table_id').innerText;"];

4. Import that text with SemanticImportString:

dataset = SemanticImportString[tableText]

This imports the text as a Dataset (partially shown below):

5. For navigation to the next page, use the CSS selector for the Next button:

WebExecute[browser, "ClickElement" -> {"CSSSelector", "#table_id_next"}]

Answer 2

The code now modified to incorporate Xminer's answer is now as follows:

page[n_] := WebExecute["ClickElement" -> Evaluate[WebExecute[session, "LocateElements" -> "CSSSelector" -> "#table_id_paginate>span>a:nth-child(" <> ToString@n <> ")"] // First]]; 
session = StartWebSession["Chrome"]; WebExecute[session, 
 "OpenWebPage" -> "https://trumpgolfcount.com/displayoutings"];
page[1]
rowheaders = WebExecute[session, "LocateElements" -> "Tag" -> "th"];
rowrecords1 = WebExecute[session, "LocateElements" -> "Tag" -> "td"];
headers = WebExecute["ElementText" -> rowheaders];
numberOfHeaders = Length[headers];
golfrecords1 = WebExecute["ElementText" -> rowrecords1];
newgolfrecords1 = Partition[golfrecords1, numberOfHeaders];
page[2]
rowrecords2 = WebExecute[session, "LocateElements" -> "Tag" -> "td"];
golfrecords2 = WebExecute["ElementText" -> rowrecords2];
newgolfrecords2 = Partition[golfrecords2, numberOfHeaders];
page[3]
rowrecords3 = WebExecute[session, "LocateElements" -> "Tag" -> "td"];
golfrecords3 = WebExecute["ElementText" -> rowrecords3];
newgolfrecords3 = Partition[golfrecords3, numberOfHeaders];
newgolfrecords = Join[newgolfrecords1, newgolfrecords2, newgolfrecords3];
dataset = Dataset[Association /@ (MapThread[Rule, {headers, #}] & /@ 
       newgolfrecords)];

This works as intended, but I have not yet figured out how to structure a function that handles those lines that require repetition of scraping and then subsequently joining, particularly when one does not have a clear idea of exactly over how many rows (web pages) these will need to be repeated, noting that the row headers need to be calculated only once for the first webpage (set of row-records).

No doubt, there is a way to do this more efficiently than accomplished so far.