With my limited knowledge I could do Programmatic data access. I hope it would be useful for other answers.
The code provided in this answer is introduced to show the procedure and the author does not hold any responsibility for its usage.
Although some days don't have images (like February 20) and there isn't a consistency in naming, below code almost find all the images to this date (except for March 20 which is named really differently).
The code below defines a function named scrapImageURLs
which returns a list of image URLs based on your second approach (opening the URL and looking for the image). Then use URLDownload
to download them on your computer and process them.
You can use "StartDate"
and "EndDate"
options to filter your requests.
ClearAll[scrapImageURLs];
Options[scrapImageURLs]:={"StartDate"->{2022,2,25},"EndDate":>Floor[AbsoluteTime[]-86400,86400]};
scrapImageURLs[OptionsPattern[]]:=Block[{startDate=Max[Floor[AbsoluteTime@OptionValue["StartDate"],86400],AbsoluteTime@{2022,2,25}],endDate=Min[Ceiling[AbsoluteTime@OptionValue["EndDate"],86400],Ceiling[AbsoluteTime[],86400]],imageURLs,imageURL={},getError=False},
imageURLs=Table[Block[{dayString=ToLowerCase@DateString[day,{"MonthName","-","DayShort"}],justDayString=ToString@DateValue[day,"DayShort"],url,response},
url=Which[day===AbsoluteTime[{2022,2,25}],
"https://www.understandingwar.org/backgrounder/russia-ukraine-warning-update-russian-offensive-campaign-assessment-february-25-2022",
day===AbsoluteTime[{2022,2,28}],
"https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-28-2022",
day<AbsoluteTime[{2022,2,28}],
"https://www.understandingwar.org/backgrounder/russia-ukraine-warning-update-russian-offensive-campaign-assessment-"<>dayString,
True,"https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-"<>dayString];
response=URLRead[url];
If[response["StatusCode"]===200,
imageURL=StringCases[response["Body"],"https://www.understandingwar.org/sites/default/files/draft"~~Shortest[___]~~((justDayString~~Shortest[___]~~"ukraine")|("ukraine"~~Shortest[___]~~justDayString))~~Shortest[___]~~".png",IgnoreCase->True],Print[" ERROR on "<>dayString<>"\n "<>url<>"\n \"StatusCode\": "<>ToString[response["StatusCode"]]];getError=True;];
If[Not@getError,
If[Length@imageURL=!=0,Print[First@imageURL];First@imageURL,Print[" No image found on "<>dayString<>"\n ",url];Nothing],Nothing]
]
,{day,Range[startDate,endDate,86400]}];
imageURLs
]
Notes:
- It uses
URLRead
which was introduced in version 11.
- Since it could fail at any moment during execution, it'll print the result/error it faced while executing and return a list of valid image URLs (for days that have an image and don't get an error).
- It assumes the image filename starts with
"draft"
and ends with ".png"
and the day and the word "ukraine"
is also included (for example March-19 has 19
in its name)
- Please save the URLs and only request for days that you don't have.
Just to avoid putting pressure on servers, here is the list of image URLs up to 2022-4-20
:
{"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTFeb25%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTFeb27%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTFeb28%2C2022_0.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch1%2C2022_0.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch2%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch3%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch4%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch5%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch6%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch7%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch8%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch9%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch10%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch11%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch12%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch13%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoT_14March2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCotMarch15%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch16%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch17%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch18%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch19%2C2022.png",
"https://www.understandingwar.org/sites/default/files/UkraineCoTMarch20%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch21%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch22%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch23%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch24%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch25%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch26%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch27%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch28%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch29%2C2022_0.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch30%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch31%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril1%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril2%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril3%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril4%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril5%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril6%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril7%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril8%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril9%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril10%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril11%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril12%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCotApril13%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril14%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril15%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril16%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril17%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril18%2C2022_0.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril19%2C2022.png"}