# Deleting missing values from a dataset

I wanted to create a dataset of all UFO sightings in April of 2018. The AssociationThread doesn't want to work, because some of the entries are missing value, what is the best way to fix it?

dataA1 = Import[
"http://www.nuforc.org/webreports/ndxe201804.html", {"HTML",
"Data"}];
dataA2 = Flatten[Rest@dataA1, 1];
dataA3 =
Dataset[dataA3]

• To be clear, this isn't about data that is Missing[], but rather Import[XXXX,{"HTML", "Data"}] giving "tables" that have rows with inconsistent lengths. – Carl Lange Mar 13 '19 at 20:52

You can solve this by importing "FullData" rather than "Data".

dataA1 = Import[
"http://www.nuforc.org/webreports/ndxe201804.html", {"HTML",
"FullData"}]
dataA2 = Flatten[Most@Rest@dataA1[[8]], 1]
dataA3 = Map[AssociationThread[dataA1[[8, 1, 1]], #] &, dataA2];
Dataset[dataA3]


What's awkward about this is that there can be many empty tables that you must sift through (as you can see, I have to get the 8th element). However, it works quite well in this case.

• Thank you, but I'm not completely sure how "Data" and "FullData" differ. Why do they exist separately? – Artem Anisimov Mar 13 '19 at 21:45
• @ArtemAnisimov From the documentation for the "HTML" import/export format: "FullData" imports: "full tabular content, including empty HTML table and list elements". "Data" doesn't import empty elements. – Carl Lange Mar 13 '19 at 22:28