6
$\begingroup$

Part 1: Difficulties using Entities

Happily realizing that there is a lot of curated data "under the hood" in Mathematica I wanted to access population data for some Administrative Divisions in Germany or more precisely in Hesse.

That is what AdministrativeDivisionData is ready made for, so let's use it:

subdivisionsHesse = AdministrativeData[
    Entity[ "AdministrativeDivision", { "Hesse", "Germany" }],
    "Subdivisions" (* property we are enquiring about *)
]

subdivisionsHesse

So great, there is Darmstadt, which is a governmental district (or province) in Hesse, and here is how many people live in that administrative subdivision:

AdministrativeDivisionData[
    Entity[ "AdministrativeDivision", { "Darmstadt", "Hesse", "Germany"}],
    "Population"
]

3741170 people

Which seems close enough to what is given for Regierungsbezirk Darmstadt.

But would I have tried the following (rather logical) piece of code to access the information I would have been grounded:

AdministrativeDivisionData[
    First @ subdivisionsHesse,
    "Population"
]

Missing[NotAvailable]

So how come? Which leads to...

Part 2: Who Cures the Curator? (Subtleties of Curated Data)

Taking a closer look at our list of subdivisions for Hesse:

subdivisionsHesse

we try this:

FullForm @ First @ subdivisionsHesse

Entity["AdministrativeDisivion", List["DarmstadtUrban","Hesse","Germany"] ]

So the curator and the curated data have somehow gotten lost in the system of administrative divisions in Germany: Below the level of the Länder (states) there are rural districts and urban districts.

So part of the riddle is solved because "DarmstadtUrban" is an urban district which unfortunately does not give "Population" values. Although the population for the city of Darmstadt is known:

CityData[ { "Darmstadt", "Hesse", "Germany" }, "Population" ]

141 257 people

But then where did the first population value come frome? Something seems to be screwed up with Hesse (everybody knows that around here :) ) or rather with the AdministrativeDivisions or more precisely the subdivisons given by Mathematica.

As you can see for Hesse, there are only three administrative provinces or governmental districts: Kassel, Gießen and Darmstadt.

But Mathematica has given five and while Kassel is listed, Gießen is not and certainly Frankfurt am Main, Offenbach and Wiesbaden do not belong there.

So maybe Mathematica has given a list of the districts of Hesse (which would be a systematic and legitimate approach)? But alas there are 21 disctricts in Hesse so that is not the solution.

But again Wikipedia helps as you can see that the above list subdivisionsHesse exactly matches the list of independent cities.

But then the whole thing is screwed:

  1. Subdivisions lists cities where districts or administrative provinces should have been given.
  2. Accordingly there are no AdministrativeDivisionData for entities of type "City".

Is there a system to this mania? And do I have to double and tripple check before using other AdministrativeDivisionData and other curated data?

$\endgroup$
  • $\begingroup$ I just checked for SchleswigHolstein where it correctly lists 11 rural districts and 4 urban districts. But try NorthRhineWestphalia and you will get 22 subdivisions which make no sense whatsoever (cf. NorthRhineWestphalia). $\endgroup$ – gwr Mar 1 '16 at 11:10
  • $\begingroup$ "Melden macht frei!" (reporting wil relieve you) as they told us in the military. So the case is passed to WRI. $\endgroup$ – gwr Mar 1 '16 at 14:01
  • 3
    $\begingroup$ the WRI interpretation of "curated" is sourced, scraped, extracted (whatever). It does not mean each datum has been manually checked which is why it should be used with caution. The accuracy of Australian data is poor. They simply do not have the man hours to actually check this stuff it seems but disappointingly seem to operate on geological timescales when fixing errors that have been brought to their attention $\endgroup$ – Mike Honeychurch Mar 1 '16 at 21:57
  • $\begingroup$ You have given a quite specific and elaborate example of the remaining difficulties of getting curated reliable data. There's no way one of the ~ 700 people (I hear?) working at WRI could have checked information that is so specific. But still the idea of this "formal, standardized, curated data Google" will surely catch on and be improved if we keep using and demanding things like this, despite the current flaws. $\endgroup$ – masterxilo Jun 26 '16 at 0:57
  • $\begingroup$ @masterxilo I totally agree with your statement, that curated data is extremely useful (in fact a usp of the Wolfram Language) and we should continue to demand it. Nevertheless, my example is specific but the mistake is a general one and holds for pretty much all AdministrativeData for Germany which is maybe not simply a "blip on the map". How can I trust such other data then? We need more tests and hopefully users keep complaining. $\endgroup$ – gwr Jun 26 '16 at 14:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.