Finding Vaccine Appointments in MA

Question

Everyday I spend a crazy amount of time clicking though each maze of hyperlinks and forms on this horrendous site: https://vaxfinder.mass.gov/

Is there anyway that Mathematica can help speed up the process of finding out if there are available appointments for people with 2+ co-morbidities?

Answer 1

Disclaimer: This answer and all of the codes created only for educational purposes and the author does not hold any responsibility.

WARNING: Since running the codes below will put heavy pressure on site and may prevent others who are really in need of the vaccine from accessing the site, PLEASE DO NOT RUN IT REGULARLY.

Base on what I see, you check all the places for availability. Mathematica has different ways to extract data from the web, this solution uses StartWebSession which introduced in version 12. basically, it will open a new browser, in this case, Firefox but you can change it to Chrome (step 3) and browse the pages automatically and will create a dataset of all the places:

1- Defined a function to grab links of individual pages:

grabLinks[session_WebSessionObject, url_ : Null] := Module[{},
  If[url != Null, WebExecute[session, "OpenPage" -> url];];
  WebExecute[session, "JavascriptExecute" -> "temp00=[];
document.querySelectorAll('a.btn.secondary').forEach(function(elem,i){
  temp00.push(elem.href);
});return temp00;"]]

2- Another function to extract table from each page:

extractInformation[session_WebSessionObject, url_String] := 
 Module[{name, response, t1},
  WebExecute[session, "OpenPage" -> url];
  name = First@WebExecute[session, "ElementText" -> "Tag" -> "h1"];
  response = 
   WebExecute[session, 
    "JavascriptExecute" -> 
     "if(document.querySelectorAll('div.table-empty').length==1){
  return 'Empty';
}else{return document.querySelector('table').innerText;}"];
  If[response == "Empty", 
   t1 = {<|"Date" -> "-", "Hours" -> "-", "Available" -> "-"|>};,
   t1 = SemanticImportString[response, {"Date", "Text", "Integer"}, 
      "NamedRows"];];
  Return[Prepend[#, "Name" -> name] & /@ t1]]

3- Start a session and open the url:

mainSession = StartWebSession["Firefox"];

WebExecute[mainSession, "OpenPage" -> "https://vaxfinder.mass.gov"];

4- Extract the number of pages and populate page links base on that:

numberOfPages = 
  StringCases[First@WebExecute[mainSession, "ElementText" -> "CSSSelector" -> "p.current"], 
     DigitCharacter ..] // Last // ToExpression;

pages = Table["https://vaxfinder.mass.gov/?page=" <> ToString@i, {i, 2, numberOfPages}];

5- Extract individual links (places) from pages:

links = Flatten[{grabLinks[mainSession], grabLinks[mainSession, #] & /@ pages}];

6- apply extractInformation on all the links and show the results:

result = extractInformation[mainSession, #] & /@ links;

Dataset @ Flatten @ result

7- exit the browser:

DeleteObject[mainSession];

As I explained in the WARNING, I didn't run it in full capacity, here are two sample outputs:

If you want to check a particular pages, just slice pages before step 5 (starts from page 2 to total pages):

(* only pages 1, 2 ,3 *)
pages = pages[[2;;3]];

(* only pages 2,3 *)
pages = pages[[2;;3]];
links = links[[26;;]]; (* do it after step 5 *)

The output is a dataset, so you can easily filter those places which have availability:

Dataset[Flatten@result][Select[#Available > 0 &]]

Full code:

grabLinks[session_WebSessionObject, url_ : Null] := Module[{},
  If[url != Null, WebExecute[session, "OpenPage" -> url];];
  WebExecute[session, "JavascriptExecute" -> "temp00=[];
document.querySelectorAll('a.btn.secondary').forEach(function(elem,i){
  temp00.push(elem.href);
});return temp00;"]]

extractInformation[session_WebSessionObject, url_String] := 
 Module[{name, response, t1},
  WebExecute[session, "OpenPage" -> url];
  name = First@WebExecute[session, "ElementText" -> "Tag" -> "h1"];
  response = 
   WebExecute[session, 
    "JavascriptExecute" -> 
     "if(document.querySelectorAll('div.table-empty').length==1){
  return 'Empty';
}else{return document.querySelector('table').innerText;}"];
  If[response == "Empty", 
   t1 = {<|"Date" -> "-", "Hours" -> "-", "Available" -> "-"|>};,
   t1 = SemanticImportString[response, {"Date", "Text", "Integer"}, 
      "NamedRows"];];
  Return[Prepend[#, "Name" -> name] & /@ t1]]


mainSession = StartWebSession["Firefox"];

WebExecute[mainSession, "OpenPage" -> "https://vaxfinder.mass.gov"];

numberOfPages = 
  StringCases[
     First@WebExecute[mainSession, 
       "ElementText" -> "CSSSelector" -> "p.current"], 
     DigitCharacter ..] // Last // ToExpression;

pages = Table["https://vaxfinder.mass.gov/?page=" <> ToString@i, {i, 2, numberOfPages}];

links = Flatten[{grabLinks[mainSession], grabLinks[mainSession, #] & /@ pages}];

result = extractInformation[mainSession, #] & /@ links;

Dataset@Flatten@result

DeleteObject[mainSession];