# Import all data from a HTML table, including empty cells

I wanted to create a dataset of all UFO sightings in April of 2018. The AssociationThread doesn't want to work, because some of the entries are missing value, what is the best way to fix it?

dataA1 = Import[
"http://www.nuforc.org/webreports/ndxe201804.html", {"HTML",
"Data"}];
dataA2 = Flatten[Rest@dataA1, 1];
dataA3 =
Dataset[dataA3]

• To be clear, this isn't about data that is Missing[], but rather Import[XXXX,{"HTML", "Data"}] giving "tables" that have rows with inconsistent lengths. Mar 13, 2019 at 20:52

To be clear, this isn't about data that is Missing[] (in which case, you would use DeleteMissing), but rather Import[XXXX,{"HTML", "Data"}] returning "tables" that have rows with inconsistent lengths.

You can solve this by importing "FullData" rather than "Data".

dataA1 = Import[
"http://www.nuforc.org/webreports/ndxe201804.html", {"HTML",
"FullData"}]
dataA2 = Flatten[Most@Rest@dataA1[[8]], 1]
dataA3 = Map[AssociationThread[dataA1[[8, 1, 1]], #] &, dataA2];
Dataset[dataA3]


What's awkward about this is that there can be many empty tables that you must sift through (as you can see, I have to get the 8th element). However, it works quite well in this case.

• Thank you, but I'm not completely sure how "Data" and "FullData" differ. Why do they exist separately? Mar 13, 2019 at 21:45
• @ArtemAnisimov From the documentation for the "HTML" import/export format: "FullData" imports: "full tabular content, including empty HTML table and list elements". "Data" doesn't import empty elements. Mar 13, 2019 at 22:28