3
$\begingroup$

I wanted to create a dataset of all UFO sightings in April of 2018. The AssociationThread doesn't want to work, because some of the entries are missing value, what is the best way to fix it?

dataA1 = Import[
"http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 
"Data"}];
dataA2 = Flatten[Rest@dataA1, 1];
dataA3 = 
 Map[AssociationThread[First[dataA1], #] &, dataA2]
Dataset[dataA3]
$\endgroup$
1
  • 1
    $\begingroup$ To be clear, this isn't about data that is Missing[], but rather Import[XXXX,{"HTML", "Data"}] giving "tables" that have rows with inconsistent lengths. $\endgroup$
    – Carl Lange
    Mar 13, 2019 at 20:52

1 Answer 1

7
$\begingroup$

To be clear, this isn't about data that is Missing[] (in which case, you would use DeleteMissing), but rather Import[XXXX,{"HTML", "Data"}] returning "tables" that have rows with inconsistent lengths.

You can solve this by importing "FullData" rather than "Data".

dataA1 = Import[
  "http://www.nuforc.org/webreports/ndxe201804.html", {"HTML", 
   "FullData"}]
dataA2 = Flatten[Most@Rest@dataA1[[8]], 1]
dataA3 = Map[AssociationThread[dataA1[[8, 1, 1]], #] &, dataA2];
Dataset[dataA3]

What's awkward about this is that there can be many empty tables that you must sift through (as you can see, I have to get the 8th element). However, it works quite well in this case.

$\endgroup$
2
  • $\begingroup$ Thank you, but I'm not completely sure how "Data" and "FullData" differ. Why do they exist separately? $\endgroup$ Mar 13, 2019 at 21:45
  • 2
    $\begingroup$ @ArtemAnisimov From the documentation for the "HTML" import/export format: "FullData" imports: "full tabular content, including empty HTML table and list elements". "Data" doesn't import empty elements. $\endgroup$
    – Carl Lange
    Mar 13, 2019 at 22:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.