Resolving my Entity* Crisis

Question

In the Wolfram Language I can usually guess a function's name, operation or idiomatic application but I don't have such a sense for various Entities ...

... instead, I seem to keep needing to consult the documentation. This suggests I'm yet to appreciate an underlying, organizing principle or abstraction; to grok it in ways that allow better evaluations of WDF's applicability. So is there a nice example/holistic way of conceptualizing/visualizing this framework?

Answer 1

As an employee at Wolfram who is partly involved with development of the entity framework, let me give my take on things. Particularly, I want to address the following points raised by the OP and in the other answers:

1) How can we cut through the clutter and find a minimal set of commands that allow us to best utilize the framework?

2) What discovery procedures exist, outside of the Ctrl-= that we all know and love, to find out what data is available and the relationships between entities, properties, and classes without resorting to reading a ton of documentation?

First, to point 1, ConvexMartian's answer is largely correct about the underlying organizational hierarchy of the system. At the highest level, we have "Domains", or more formally "EntityTypes", which comprise a group of conceptually related entities and properties. The available entity types can be discovered through the simple command

EntityValue[]

The list that we get as the result tells us, for example, that "Planet" is an available entity type (one which I'll use in my examples to follow). This simple example also brings me to my first point that may start to clear away some of the clutter: it is, when in doubt, EntityValue is the main function one should be using to get information about the entity world. All other of the "***Entity***" named symbols should be thought of as subsidiary to EntityValue, repackagings of things that can be done through EntityValue, or things that can be passed into EntityValue as arguments. This isn't always the case, and I'll talk about some specifics in a bit, but it's a good rule of thumb, and the EntityValue refpage is always a good place to start when you're wondering how things work in the Entity world.

So, once you've found the domain you want to study, what general information can you get about it? Again, we use EntityValue:

EntityValue["Planet", "Entities"]
EntityValue["Planet", "Properties"]
EntityValue["Planet", "EntityClasses"]
EntityValue["Planet", "PropertyClasses"]
EntityValue["Planet", "XXXXX"]...

all do what one expects (although for "PropertyClasses", you get back an empty list, as the "Planet" domain doesn't have any defined property classes). The other allowed second arguments for EntityValue[domain, _] are shown on the EntityValue ref page, along with descriptions.

The usual way one wants to use the entity framework is to get at actual data values for a particular Entity/Property pair, and once again, EntityValue is the way to do this (thus the "Value" in the name). So, for example, if I want to know the mass of Mars, I can do

EntityValue[Entity["Planet", "Mars"], EntityProperty["Planet", "Mass"]]
(* or, more simply, *)
EntityValue[Entity["Planet", "Mars"], "Mass"]

Things to note:

• The commands Entity and EntityProperty (not to mention EntityClass and EntityPropertyClass) serve really as wrappers for arguments that get fed into EntityValue.
• They always take the string name for the domain as their first argument, and the so-called "CanonicalName" of the Entity, Property, or Class you are talking about as the second argument.
• They are essentially the InputForm of the yellow boxes you see the front end returning in Entity/Property queries.
• EntityValue can also take a LIST of Entities/Properties for either/both of the first and second arguments. Doing this can be greatly more efficient, due to backend database voodoo, than, say, mapping over these lists. EntityValue can also take a so-called "annotation" as a third argument, the most useful application of which may be in using the "XXXAssociation" annotations when data is requested about listed entities/properties, since it allows these data to be returned in a nicely packaged Association form. Examples include "EntityAssociation", "PropertyAssociation", "EntityPropertyAssociation", and the undocumented "NonMissingEntityAssociation" and "NonMissingPropertyAssociation"
• so-called "qualifiers" allow you to further specify a property, allowing some neat tricks. They are passed, as a list, to the third argument of EntityProperty. So, for example, if you want the population of France in 1970 you could do EntityValue[Entity["Country", "France"], EntityProperty["Country", "Population", {"Date" -> 1970}]]. Available qualifiers can be discovered via e.g. EntityValue[EntityProperty["Country", "Population"], "Qualifiers"].

We can also use EntityValue to get at the same information returned by some of the more specialized, but less generally useful, commands mentioned by the OP, thereby freeing up brain space by allowing us to avoid memorizing several different syntaxes. Thus, for example, the commands

EntityClassList["Planet"]
EntityList[EntityClass["Planet", "InnerPlanet"]]
RandomEntity["Planet", 5]
Entity["Planet", "Mars"]["Mass"]

can be replaced with

EntityValue["Planet", "EntityClasses"]
EntityValue[EntityClass["Planet", "InnerPlanet"], "Entities"]
EntityValue["Planet", {"RandomEntities", 5}]
EntityValue[Entity["Planet", "Mars"], "Mass"]

respectively. In preferring the latter versions, the added consistency in the syntax should hopefully contribute to a better ability to remember the right command when it's need, or at least to having a "one-stop shop" in the documentation to go to for help.

Let me now touch, in turn, on each of the commands listed in the OP's screen shot that I have not already mentioned, and give a quick insider's view on if/when/how they should be used.

• EntityTypeName: not particularly useful. I hadn't even heard of it until now. If the EntityType of an orange Entity box is needed and not already known, just apply InputForm to it and take the first Part.

• EntityCopies/EntityGroup: these may be useful for passing into EntityValue if you want to get information on an aggregation of Entities, but there are other, equally efficient if less syntactically elegant, ways of doing this (e.g. get the data for a simple list of Entities and then apply the desired multiplication/addition/or other aggregation yourself). I personally wouldn't devote the mental overhead to thinking about these commands unless you do a lot of work with entity aggregations.

• EntityInstance allows some really interesting functionality, a lot of which is still being expanded upon, and would take too long to explain to go into here. But if you're able to grok from the docs what it does at the detailed level and would find it useful, then in the big picture, just think of treating an EntityInstance as an entity. It will work in the first argument of EntityValue in almost exactly the same way.
• GeoEntities/$TimeZoneEntity are very specialized constructs only tangentially connected with the "core" Entity world functions. See the docs if you think you have use for them (their names are indicative of the general nature of the thing they do), but I would guess that 99% of users will never need to use them.
• To/FromEntity: These are used to convert between a non-Entity Wolfram Language expression and a corresponding (as judged by us) equivalent Entity (ToEntity goes one way, FromEntity goes the other). So, for example, ToEntity[Red] returns an Entity in the "Color" domain. Round-tripping doesn't always result in the original input, and currently the only supported domains are "Color", "WolframLanguageSymbol", "MathematicalFunction", and "Graph". So these commands can be somewhat useful, and can serve as a bit of programmatic Ctrl-= for the Entity world, but their definitions would need to be extended before they become maximally useful.

A final point:
There is some confusion, not all of it unjustified, about the relationship between EntityValue domains and the corresponding XXXData symbols in the Wolfram Language. So, for example, we can use EntityValue["Planet", ...], but we can equivalently use PlanetData[...]. The argument structure for these two methods is somewhat different. Furthermore, sometimes entities or properties get renamed or omitted between the two worlds (e.g., we have EntityProperty["Country", "ElectricityTotal"], but CountryData[..., "ElectricityProduction"]). Furtherfurthermore, even the entity domain name itself does not always correspond with the XXXData symbol name (compare WolframLanguageData <---> EntityValue["WolframLanguageSymbol", ...], AnatomyData <---> EntityValue["AnatomicalStructure", ...]). To address this, I will merely say that EntityValue is the preferred way of accessing the data, has only one syntax to learn, and offers data on more domains than exist as dedicated XXXData symbols. That being said, the more rigid structure of the EntityValue world prevents us, currently, from presenting data on many domains that do have an XXXData symbol (e.g., "WeatherData", "WikipediaData", ...). For a complete list of all available XXXData functions, one can evaluate

First[StringSplit[#, "Data"]] & /@ Names["*Data"]

Hope all this helps!

Answer 2

From what I can tell, the Wolfram Knowledge Base is organized around three ideas from epistemology: concepts, units, and properties:

A concept is a set of related concrete perceptions. For example, 'stone' is the concept that represents all the physical stones in the world. Each physical stone that exists is called a unit of the concept 'stone'. Grouping units into a concept makes them easier to think about and talk about.

For example, in

Entity["Planet", "Mercury"]

"Planet" is the concept and "Mercury" is a unit of that concept.

For example, using the free-form input (Ctrl-=) and typing "new york" results in the Entity

 Entity["City", {"NewYork", "NewYork", "UnitedStates"}]

Here, "City" is the concept and "New York" is the unit. (Additional data is needed to disambiguate "New York" -- we're talking about the city in New York state, in the United States.)

Every concept possesses a list of properties shared by all units of the concept. E.g., the concept 'stone' has the properties: shape, color, weight, rock type, etc, because each individual real-world stone has these properties.

For example, here are the properties shared by every unit of the "Planet" concept:

 Entity["Planet"]["Properties"]

 (* absolute magnitude H, age, albedo, ... *)

Here are the properties that every "City" has:

 Entity["City"]["Properties"]

 (* administrative region, number of aggravated assaults, rate of aggravated assault, ... *)

Under the hood, a concept's property is represented as an expression like this:

 EntityProperty["City", "AggravatedAssaultRate"]    

For any unit of a concept, these properties take specific values: the stone near my house is oblong, brown, 1.2 lb, and made of granite.

For example, here is the specific value of the "age" property of the unit "Mercury":

 Entity["Planet", "Mercury"]["Age"]

 (* 4.5 * 10^9 years *)

Here is the value of the "aggravated assult rate" property of the "New York" unit:

 Entity["City", {"NewYork", "NewYork", "UnitedStates"}]["AggravatedAssaultRate"]

 (* 0.0031493 crimes/(person yr) *)

That seems to be the basic setup of the Wolfram Knowledge Base.

Extensions

A class is a named subset of units of a concept:

  EntityClass["City","UnitedStatesCapitals"]  (* a subset of cities *)

  EntityList[%]

  (* Montgomery, Juneau, Pheonix, Little Rock, ... *)

A property class is a subset of properties of a concept:

 EntityPropertyClass["Chemical", "ToxicityProperties"]  (* a subset of properties of the concept of "chemical" *)

 EntityProperties[%]

 (* odor, odor threshold, ... *)

I learned this epistemology from Lecture 7 of Leonard Peikoff's Introduction to Logic audio lectures. ($23, here)

Answer 3

Honestly, most of the time I just see and copy what the natural language input does.

Suppose I want to know the heights of some buildings. I could look through the documentation. Or I could just give a natural language example and see what is produced.

Now I know that I can query for the height of building entities. So I can get the heights of the buildings I wanted:

buildings = 
 Interpreter["Building"][{"Taj mahal", "Chrystler Building", "Big Ben"}]

#[EntityProperty["Building", "Height"]] & /@ buildings

I understand that this isn't really "groking it", but the relationships between entities and what properties they might have is necessarily very complicated and messy. Being able to access this by natural language is important. It's kinda the only sane way unless you want to read endless documentation about the different entities and their properties and relations to each other.