2
$\begingroup$

I would like to implement a function to import several dataset from the same website source, where for each dataset it changes just one word on the website address.

Here the website: https://statistichecoronavirus.it/regioni-coronavirus-italia/.../ where ... represents the part to change in order to get each dataset. I would like to have the following variable vectors from each webpage:

total infected
total active cases 
total deaths
total healed
daily increment infected
daily incremental active cases
daily increment deaths
daily increment healed

I tried to write the following, but it doesn't work:

covid19[region_] := 
 Module[{data, process, date, infected, cases, deaths, healed},
  data = Import[
    "https://statistichecoronavirus.it/regioni-coronavirus-italia/"<>region <> "/", "Data"];
  process[s_String] := Module[{d = StringSplit[s, " "][[1]]}, 
    If[StringContainsQ[d, "/"], 
     DateObject[{d <> "/2020", {"Day", "/", "Month", "/", "Year"}}], 
     ToExpression[StringReplace[d, "." -> ""]]]];
  process[s_Integer] := s (*to process the last string*);
  date = Map[process, data[[2, All, ;; -2]], {2}][[All, 1]];
  infected= Map[process, data[[2, All, ;; -2]], {2}][[All, 2]];
  cases = Map[process, data[[2, All, ;; -2]], {2}][[All, 3]];
  deaths = Map[process, data[[2, All, ;; -2]], {2}][[All, 4]];
  healed = Map[process, data[[2, All, ;; -2]], {2}][[All, 5]];
  <|region -> <|"Infected" -> infected, "Cases" -> cases, "Deaths" ->deaths, "Healed" -> healed|>|>]
status = covid19 /@ {"Lombardia", "Sicilia"} // Association;

That's what I've done without functional programming bases:

ClearAll;
data = Import["https://statistichecoronavirus.it/coronavirus-italia/",
    "Data"];
Table[Drop[%[[2, i]], -1], {i, 1, Length[%[[2]]] - 1}];
temp = Table[StringReplace[%[[i]], "." -> ""], {i, 1, Length[%]}];
StringExtract[##, 1] & /@ temp;
Join[{data[[1, 2 ;; All]]}, 
  ToExpression[%[[All, 2 ;; All]]], {ToExpression[
    Drop[data[[2, -1]], -1][[2 ;; All]]]}];
tot = Transpose[
   Insert[% // Transpose, 
    Flatten[{"Data", StringExtract[##, 1] & /@ temp[[All, 1]], 
      "20/02"}], 1]];
contagi = tot[[2 ;; All, 2]];
casi = tot[[2 ;; All, 3]];
morti = tot[[2 ;; All, 4]];
guariti = tot[[2 ;; All, 5]];
Join[{data[[1, 2 ;; All]]}, 
  ToExpression[
   ArrayReshape[
    StringCases[StringJoin[temp], 
     Shortest["(+" ~~ x__ ~~ ")"] -> x], {Length[temp], 
     4}]], {ToExpression[Drop[data[[2, -1]], -1][[2 ;; All]]]}];
par = Transpose[
   Insert[% // Transpose, 
    Flatten[{"Data", StringExtract[##, 1] & /@ temp[[All, 1]], 
      "20/02"}], 1]];
\[CapitalDelta]contagi = tot[[2 ;; All, 2]];
\[CapitalDelta]casi = tot[[2 ;; All, 3]];
\[CapitalDelta]morti = tot[[2 ;; All, 4]];
\[CapitalDelta]guariti = tot[[2 ;; All, 5]];
date = Table[
  DateObject[{tot[[i, 1]] <> "/2020", {"Day", "Month", "Year"}}], {i, 
   2, Length[tot]}]
G1 = TemporalData[contagi, {date}];
G2 = TemporalData[casi, {date}];
G3 = TemporalData[morti, {date}];
G4 = TemporalData[guariti, {date}];
Show[ListLinePlot[G1], ListLinePlot[G2], ListLinePlot[G3], 
 ListLinePlot[G4]]
Interpolation[G1, InterpolationOrder -> 4]

My final goal is to get several interpolation and bestfit function for each region. I would like to avoid repeating it for each "region".

$\endgroup$
3
$\begingroup$

Update Plot data for several regions

provinces = {"lombardia", "sicilia"};
dataByProvince = covid19Italy /@ provinces;

dataByProvince //
  Map[(KeyTake[#, {"date", "total infected"}] &), #, {2}] & //   
  Values //
  Flatten[#, 1] & //
  DateListPlot[#,
   ScalingFunctions -> "Log",
   PlotLegends -> Capitalize@provinces,
   PlotMarkers -> Automatic,
   PlotTheme -> "Detailed",
   PlotLabel -> Style["COVID-19 Italy - Total Infected by Province", 14, Black]] &

enter image description here


That code looks quite complicated. Here is an alternative (and I think, simpler) implementation.

covid19Italy[region_] := 
 Module[{header, convertDelta, insertZero, data, processedData},

  header = {"date", "total infected", "total infected change", 
    "total deaths", "total deaths change", "total healed", 
    "total healed change"};

  insertZero[n_] := ToString[n] <> " (+0)";

  convertDelta[s_] := 
   StringReplace[s, {"(" -> "", ")" -> ""}] // StringSplit // 
    ToExpression;

  data = Import["https://statistichecoronavirus.it/regioni-coronavirus-italia/" <> 
      region <> "/", "Data"][[2, 2]];

  processedData = 
   data //
       MapAt[StringReplace[" In aggiornamento" -> ""] /*
         (DateObject[{# <> "/2020", {"Day", "/", "Month", "/", "Year"}}] &), #, {All, 1}] & //                 
      MapAt[insertZero, #, {-1, 2 ;;}] & //
     MapAt[convertDelta, #, {All, 2 ;;}] & //
    Map[Flatten];
  <|region -> (AssociationThread[header -> #] &) /@ processedData|>]

lombardia = covid19Italy["lombardia"];

lombardia // Map[KeyTake[#, {"date", "total infected"}] &] // DateListPlot

enter image description here

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ You might also want to take a look at this $\endgroup$ – Rohit Namjoshi Mar 26 at 20:26
  • $\begingroup$ Really Rohit, thank you, for the brilliant solution and the upmentioned link also. I must confess that I can't understand perfectly all the local steps of the covid19Italy function and your solutions are for me the best possible case study to learn programming in Mathematica environment. I think to comparise several region DateListPlots with Show function; should I already assign different colors within DateListPlot function (I don't know how in this "//" form) or can I assign it within Show function (I tried in this way but it doesn't works)? $\endgroup$ – Mick Mar 26 at 20:58
  • 1
    $\begingroup$ Added an example showing how to plot data for multiple provinces. $\endgroup$ – Rohit Namjoshi Mar 28 at 2:20
  • $\begingroup$ Rohit, could you help me please? mathematica.stackexchange.com/questions/218408/… $\endgroup$ – Mick Apr 4 at 16:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.