Animating battlefield data map of war in Ukraine

Question

UPDATE on FINAL RESULT

Thanks to @SquareOne effort I generated higher-resolution videos with smoothing transitions that can be seen here:

• https://www.linkedin.com/feed/update/urn:li:activity:6926902980323512320/

• https://twitter.com/superflow/status/1521191832012705792

I might post my version of @SquareOne code with some bug corrections later. I am grateful to this community and @SquareOne for outstanding support.

INTRO & BOUNTY TARGET

Dear friends, as you know there is currently an ongoing war in Ukraine: https://war.ukraine.ua

I need your help on some coding in image/video processing, which is very simple to formulate, but not obvious in execution. Smooth exactly-timed transitions between video frames and perfect alignment of video frames is a key challenge here. BOUNTY TARGET IS EXPLAINED AT THE END OF THE POST.

DATA Description

United States' Institute for the Study of War (ISW, "a non-partisan, non-profit, public policy research organization") performs daily research and publishes daily maps of the battlefield. Their work is public and gives many references to data sources they use. For instance...

The whole-Ukraine overview map from the 1st day of invasion on FEB 24, 2022 (source):

A recent whole-Ukraine overview map from APR 19, 2022 (source):

DATA Source

These maps are published almost daily with all publications gathered here: https://www.understandingwar.org/publications

BOUNTY TARGET

BOUNTY WILL BE AWARDED TO the CODE GENERATING BEST .MP4 VIDEO of a SEQUENCE of MAPS.

Basic part for the BOUNTY:

• Programatic data access. While URLs of daily articles and images follow some pattern, it is not always regular. How do we write a piece of code that accesses whole-map programmatically? We do not want do do this manually. Approach 1: look at the daily image URLs (example), but they are still not regular. Approach 2: look at the daily articles URLs and get 1st image from the article (example), but they are still not regular. Maybe there are other approaches. START DAT: FEB 24. END DATE: CURRENT DAY.

• Each frame must have a date stamp. For example: FEB 24, 2022, FEB 25, 2022, etc.

• Image alignment of Ukraine border - the GREATEST challenge. All these maps-images are slightly different. Ukraine country border should NOT jump from frame to frame.

• Duration of a frame and smoothness of transition Each map-image (key-frame) should be held on screen 1 second. Each of 3 transitional blended frames should last 0.15 seconds. This is a toy example of how to achieve it. Imagine you have just 3 key-frames.

Build transitions via interpolated blended frames:

frames=Values[TimeSeriesResample[TimeSeries[imglist,{0}],1/4]]

Define non-inform timings as

In[68]:= timings=Flatten[Riffle[Table[1,3],{Table[.15,3]}]]

Out[68]= {1,0.15,0.15,0.15,1,0.15,0.15,0.15,1}

Create video as

SlideShowVideo[frames -> timings]

Export to .MP4 via

http://reference.wolfram.com/language/ref/format/MP4.html

Thank you very much for considering this!!! Collecting data from independent sources, and displaying it in a comprehensive animation, can help to inform society in ways that numbers and unorganized static images cannot. This is the sort of thing we as a community can do best in these difficult times.

Answer 1

Here's a first pass at naively grabbing and animating the main map frames:

april = Table[
dayImgs = Import["https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-april-"<>ToString[n],"Images"];
img = dayImgs[[5]]; (* brittle *)
ymax=Max@ImageValuePositions[Image@ImageData[img][[All,{10,10}]],{0.9686274509803922`,0.9686274509803922`,0.9686274509803922`,1.`}][[All,2]];
ymin=Max@ImageValuePositions[Image@ImageData[img][[All,{10,10}]],{0.,0.,0.,1.}][[All,2]];
ImageTrim[img,{{2550,ymax},{0,ymin}}],
{n,1,19}]

ListAnimate[april]

Or you can try:

SlideShowVideo[Thread[april->1]]

Update:

Another approach could be to try registering with GeoGraphics and using color analysis to get "the locations", here's a sketch:

dayImgs = Import["https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-april-18","Images"];
img = dayImgs[[5]]; (* same as above *)
ymax=Max@ImageValuePositions[Image@ImageData[img][[All,{10,10}]],{0.9686274509803922`,0.9686274509803922`,0.9686274509803922`,1.`}][[All,2]];
ymin=Max@ImageValuePositions[Image@ImageData[img][[All,{10,10}]],{0.,0.,0.,1.}][[All,2]];
i=ImageTrim[img,{{2550,ymax},{0,ymin}}]

(* Extract the legend colors and overlay graphics *)
{rc,uc}={RGBColor[0.9568629185428273, 0.6901959859893049, 0.6862744789494377, 1.], 
 RGBColor[0.6666667137194934, 0.9137255348660686, 0.9882353221073724, 1.]};
{ru,ua}=Image[ColorReplace[Image[DeleteSmallComponents[Binarize[Blur[ColorDetect[i, ColorsNear[#, 0]], 6], .1]]],
{White->#, Black->Transparent}],ImageSize->{613.7,650.6}]&/@{rc,uc};

g=GeoGraphics["World", 
GeoRange->{{38.5, 53.89}, {24.69, 39.5}},ImageSize->{613.7,650.6}, (* guesstimates *)
GeoProjection->"Mercator", GeoBackground->"CountryBorders"];
Magnify[Row[{Overlay[{g,ru,ua}],i}],.5]

Answer 2

With my limited knowledge I could do Programmatic data access. I hope it would be useful for other answers.

The code provided in this answer is introduced to show the procedure and the author does not hold any responsibility for its usage.

Although some days don't have images (like February 20) and there isn't a consistency in naming, below code almost find all the images to this date (except for March 20 which is named really differently).

The code below defines a function named scrapImageURLs which returns a list of image URLs based on your second approach (opening the URL and looking for the image). Then use URLDownload to download them on your computer and process them.

You can use "StartDate" and "EndDate" options to filter your requests.

ClearAll[scrapImageURLs];

Options[scrapImageURLs]:={"StartDate"->{2022,2,25},"EndDate":>Floor[AbsoluteTime[]-86400,86400]};

scrapImageURLs[OptionsPattern[]]:=Block[{startDate=Max[Floor[AbsoluteTime@OptionValue["StartDate"],86400],AbsoluteTime@{2022,2,25}],endDate=Min[Ceiling[AbsoluteTime@OptionValue["EndDate"],86400],Ceiling[AbsoluteTime[],86400]],imageURLs,imageURL={},getError=False},

imageURLs=Table[Block[{dayString=ToLowerCase@DateString[day,{"MonthName","-","DayShort"}],justDayString=ToString@DateValue[day,"DayShort"],url,response},

url=Which[day===AbsoluteTime[{2022,2,25}],
"https://www.understandingwar.org/backgrounder/russia-ukraine-warning-update-russian-offensive-campaign-assessment-february-25-2022",

day===AbsoluteTime[{2022,2,28}],
"https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-28-2022",

day<AbsoluteTime[{2022,2,28}],
"https://www.understandingwar.org/backgrounder/russia-ukraine-warning-update-russian-offensive-campaign-assessment-"<>dayString,


True,"https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-"<>dayString];

response=URLRead[url];

If[response["StatusCode"]===200,
imageURL=StringCases[response["Body"],"https://www.understandingwar.org/sites/default/files/draft"~~Shortest[___]~~((justDayString~~Shortest[___]~~"ukraine")|("ukraine"~~Shortest[___]~~justDayString))~~Shortest[___]~~".png",IgnoreCase->True],Print["   ERROR on "<>dayString<>"\n   "<>url<>"\n   \"StatusCode\": "<>ToString[response["StatusCode"]]];getError=True;];

If[Not@getError,
If[Length@imageURL=!=0,Print[First@imageURL];First@imageURL,Print["   No image found on "<>dayString<>"\n   ",url];Nothing],Nothing]

]
,{day,Range[startDate,endDate,86400]}];

imageURLs

]

Notes:

• It uses URLRead which was introduced in version 11.
• Since it could fail at any moment during execution, it'll print the result/error it faced while executing and return a list of valid image URLs (for days that have an image and don't get an error).
• It assumes the image filename starts with "draft" and ends with ".png" and the day and the word "ukraine" is also included (for example March-19 has 19 in its name)
• Please save the URLs and only request for days that you don't have.

---

Just to avoid putting pressure on servers, here is the list of image URLs up to 2022-4-20:

{"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTFeb25%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTFeb27%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTFeb28%2C2022_0.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch1%2C2022_0.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch2%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch3%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch4%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch5%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch6%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch7%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch8%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch9%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch10%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch11%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch12%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch13%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoT_14March2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCotMarch15%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch16%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch17%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch18%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch19%2C2022.png",
"https://www.understandingwar.org/sites/default/files/UkraineCoTMarch20%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch21%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch22%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch23%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch24%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch25%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch26%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch27%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch28%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch29%2C2022_0.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch30%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTMarch31%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril1%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril2%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril3%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril4%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril5%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril6%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril7%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril8%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril9%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril10%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril11%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril12%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCotApril13%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril14%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril15%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril16%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril17%2C2022.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril18%2C2022_0.png",
"https://www.understandingwar.org/sites/default/files/DraftUkraineCoTApril19%2C2022.png"}

Answer 3

Here is a link to a mp4 video (400 pixel width without fade transition due to memory limit in the the free basic Wolfram Cloud)

1. Fetching

The idea is to search all the ukraine-project pages for the links to all the reports containing the maps images. The name of the reports and of the images are searched given some keywords-string patterns.

All the images are downloaded locally (these are big images which need resizing).

webpage="https://www.understandingwar.org/project/ukraine-project";
pagePattern="?page=";
titlePattern="russian-offensive-campaign";
imageTitlePattern="UkraineCo"~~__~~".png";
maxpages= Import[webpage,"Hyperlinks"] // Select[StringContainsQ[pagePattern]] //StringCases[pagePattern~~d:DigitCharacter..->d] // Flatten//ToExpression // Max;

allReportsLinks=
Range[0,maxpages] // 
Map[(
Import[webpage<>pagePattern<>ToString@#, "Hyperlinks"]//
Select[StringContainsQ[titlePattern]]
)&
]//Flatten//Union;

allImagesLinks=allReportsLinks//
Map[{#,
Import[#,"Hyperlinks"]//Select[ StringContainsQ[imageTitlePattern]]//Union}&];

URLDownload[Select[allImagesLinks,Length@#[[2]]>0&]//#[[All,2,1]]&,"."]; 

2. Resizing

Once downloaded locally, the original maps are resized for convenience and are also directly renamed with the corresponding map date.

fn=FileNames["*Ukr*.png"];
imageWidth=300;
months2num={"April"->"04","March"->"03","Feb"->"02"};
filedatePatterns={(d:DigitCharacter..)~~m:("April"|"March"|"Feb"),
(LetterCharacter~~(m:"April"|"March"|"Feb")~~d:DigitCharacter..)};

allimagefiles=
Table[fn//Map[{#,StringCases[#,pattern:>
StringJoin[m/.months2num,If[StringLength@d==1,"0"<>d,d],"_",ToString@imageWidth,".png"]]}&],
{pattern,filedatePatterns}]// Map[Select[Length@#[[2]]==1&]]//Join@@#&//Map[Flatten];

Table[img=Import[imf[[1]]];
If[Head[img]===Image,Export[imf[[2]],ImageResize[img,imageWidth]]],{imf,allimagefiles}]; 

3. Aligning

The maps are aligned according to a caracteristic pattern which is always visible on all maps.

This image pattern is a small portion of the black sea on the east side with also two country borders:

The best results are obtained when extracting and comparing the edges of the images:

imgpatt=Import["0225_300.png"]//RemoveAlphaChannel//
ImageTake[#,{280,360},{250,295}]& //EdgeDetect[#,1]&

ImageCorrelate is used here to find the positions where the image pattern best matches the maps. It is then be possible to center all the maps onto this position-pattern.

bestpos[img_,pattern_]:= ImageCorrelate[img, pattern, 
NormalizedSquaredEuclideanDistance, Padding -> None] //
ColorConvert[#,"Graylevel"]& // ImageData // Position[#, Min[#]]&//
{#[[1,2]],ImageDimensions[img][[2]]-#[[1,1]] -ImageDimensions[pattern][[2]]}&


imageWidth=300;
finalfiles=FileNames["*_"<>ToString[imageWidth]<>".png"];
res=Table[img=Import[file]//RemoveAlphaChannel;xy=bestpos[EdgeDetect[img,1],imgpatt];{xy,ImageDimensions@img-xy},{file,finalfiles}];
{xg,yb,xd,yh}=Transpose@res//Map[Transpose]//Flatten[#,1]&//Map[Max];
i0=Image[Graphics[{White,Rectangle[{0,0},{xg+xd,yb+yh}]}],ImageSize->{xg+xd,yb+yh}];

imgs=Table[img=Import[file]//RemoveAlphaChannel;
ImageCompose[i0,img,{xg,yb},bestpos[EdgeDetect[img,1],imgpatt]],{file,finalfiles}];

4. Movie

Assembling the images into a movie as indicated by @Vitaly:

frames=Values[TimeSeriesResample[TimeSeries[imgs,{0}],1/3]];
timings=Flatten[Riffle[Table[0.30,Length@finalfiles],{Table[.15,2]}]];
Remove[imgs];
SlideShowVideo[frames -> timings]

---------------------------Previous

Just a proof of concept with 4 maps, i have not finished yet. I tried to align the full images with different approaches, ImageCorrelate seems to give here the more robust results.