2
$\begingroup$

I wanted to create a dataset of all UFO sightings in April of 2018. The AssociationThread doesn't want to work, because some of the entries are missing value, what is the best way to fix it?

dataA1 = Import[
"http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 
"Data"}];
dataA2 = Flatten[Rest@dataA1, 1];
dataA3 = 
 Map[AssociationThread[First[dataA1], #] &, dataA2]
Dataset[dataA3]
$\endgroup$
  • 1
    $\begingroup$ To be clear, this isn't about data that is Missing[], but rather Import[XXXX,{"HTML", "Data"}] giving "tables" that have rows with inconsistent lengths. $\endgroup$ – Carl Lange Mar 13 at 20:52
6
$\begingroup$

You can solve this by importing "FullData" rather than "Data".

dataA1 = Import[
  "http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 
   "FullData"}]
dataA2 = Flatten[Most@Rest@dataA1[[8]], 1]
dataA3 = Map[AssociationThread[dataA1[[8, 1, 1]], #] &, dataA2];
Dataset[dataA3]

What's awkward about this is that there can be many empty tables that you must sift through (as you can see, I have to get the 8th element). However, it works quite well in this case.

$\endgroup$
  • $\begingroup$ Thank you, but I'm not completely sure how "Data" and "FullData" differ. Why do they exist separately? $\endgroup$ – Artem Anisimov Mar 13 at 21:45
  • 2
    $\begingroup$ @ArtemAnisimov From the documentation for the "HTML" import/export format: "FullData" imports: "full tabular content, including empty HTML table and list elements". "Data" doesn't import empty elements. $\endgroup$ – Carl Lange Mar 13 at 22:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.