I am reading Covid data from an external database like so:
usDailyTracking =
Import["https://covidtracking.com/api/v1/states/daily.json",
"RawJSON"] /. Null -> Missing[] // Dataset;
Those data have the problem that, when one of the entries is zero (number of positives, or number of deaths, etc.) then the corresponding entry is simply left out rather than listed as "0". So, now I would like to create a dataset that extracts certain subsets of items while replacing missing entries with zeroes. What I have is this:
Clear[toDate];
toDate[num_Integer] :=
DateObject@{Quotient[num, 10000],
Quotient[num - Quotient[num, 10000] 10000, 100], Mod[num, 100]}
usDaily =
usDailyTracking[
Table[i, {i, 1, 10}], <|"date" -> toDate[#"date"],
"State" -> #"state", "positive" -> #"positive",
"negative" -> #"negative", "posNeg" -> #"posNeg",
"death" -> #"death"|> &,
FailureAction -> {"Replace", <|"date" -> toDate[#"date"],
"State" -> #"state", "positive" -> #"positive",
"negative" -> #"negative", "posNeg" -> #"posNeg",
"death" -> 0|> &}];
So the idea would be that, whenever there's a record that does not have an item for "death", a zero is inserted for the missing item. However, with the code above I get this:
date State positive negative posNeg death
Thu 23 Apr 2020 AK 337 11824 12161 9
Thu 23 Apr 2020 AL 5778 46863 52641 197
Thu 23 Apr 2020 AR 2465 29125 31590 45
toDate[#date] #state #positive #negative #posNeg 0
Thu 23 Apr 2020 AZ 5769 52928 58697 249
Thu 23 Apr 2020 CA 37369 444728 482097 1469
Thu 23 Apr 2020 CO 10878 39767 50645 508
Thu 23 Apr 2020 CT 23100 48397 71497 1639
Thu 23 Apr 2020 DC 3361 12569 15930 139
Thu 23 Apr 2020 DE 3308 13604 16912 92
Clearly, what I would want is leave the existing items in place, and only replace the missing "death" item, so the above doesn't quite do the job.
Can someone help me fix this?
As a follow-up question, using the FailureAction approach, all I can do is detect that "something is wrong", and if so take one specific action. In a more general case, I would want to figure out which one of the items causes the failure, and then replace only the missing item with a zero. What would be the path to achieve this?
