2
$\begingroup$

I am having difficulty understanding the difference between Dataset with strings as keys versus using symbols as keys.

Below is a very simple toy dataset (my actual dataset is much larger and complex) where the keys are strings

datasetString = Dataset[{
   <|"a" -> 1, "b" -> "source1"|>,
   <|"a" -> 2, "b" -> "source2"|>,
   <|"a" -> 3, "b" -> "none"|>
   }]

Mathematica graphics

Using an example from the documentation I extract the rows where a < 3

datasetString[Select[#a < 3 &]]

Mathematica graphics

Everything works fine.

Now repeat the experiment but use symbols rather than strings for the keys.

datasetSymbol = Dataset[{
   <|a -> 1, b -> "source1"|>,
   <|a -> 2, b -> "source2"|>,
   <|a -> 3, b -> "none"|>
   }]

Mathematica graphics

The first thing I note is that the display is different.

Secondly I am unable to use the same selection process

datasetSymbol[Select[#a < 3 &]]

Mathematica graphics

Since I have access to the column using

datasetSymbol[All, Key[a]]

Mathematica graphics

I tried

datasetSymbol[Select[#[Key[a]] < 3 &]]

Here no error occurred but it returned an empty dataset.

$\endgroup$
  • 2
    $\begingroup$ What you really need is dataset[Select[#[a] < 3 &]] or dataset[Select[#[[Key [a]]] < 3 &]]. This is not a problem of dataset, which has in this respect the same semantics as associations. The syntax #a is equivalent to #["a"], not #[a], so there is an auto-stringification happening here. $\endgroup$ – Leonid Shifrin Aug 28 '17 at 17:02
  • $\begingroup$ @LeonidShifrin datasetSymbol[Select[#[a] < 3 &]] works as does datasetString[Select[#["a"] < 3 &]]. I guess the question amounts to what is the black magic as to why datasetString[Select[#a < 3 &]] works. $\endgroup$ – Jack LaVigne Aug 28 '17 at 17:12
  • 1
    $\begingroup$ @LeonidShifrin Also why do the two display differently? $\endgroup$ – Jack LaVigne Aug 28 '17 at 17:14
  • $\begingroup$ Re: why datasetString[Select[#a < 3 &]] works - this is because of the auto-stringification. The #a gets interpreted as #["a"] (try #a // FullForm), by the parser. Re: Also why do the two display differently? - this is due to the Dataset type system. Try TypeSystem`DeduceType@{<|a -> 1, b -> "source1"|>, <|a -> 2, b -> "source2"|>, <|a -> 3, b -> "none"|>} vs TypeSystem`DeduceType@{<|"a" -> 1, "b" -> "source1"|>, <|"a" -> 2, "b" -> "source2"|>, <|"a" -> 3, "b" -> "none"|>}, you see general Assoc type in the first case and more specific Struct in the second. $\endgroup$ – Leonid Shifrin Aug 28 '17 at 19:22
  • 1
    $\begingroup$ @LeonidShifrin Kindly validate the answer I posted for accuracy. Feel free to edit. I am unable to locate documentation on the TypeSystem and do not know what the difference is between Struct and Assoc but can guess. $\endgroup$ – Jack LaVigne Aug 29 '17 at 14:11
3
$\begingroup$

Part 1 - Syntax

The answer is that for Associations and Datasets keys that are strings have some special shortcut syntax compared to non-string keys.

Here are the two datasets:

datasetString = Dataset[{
   <|"a" -> 1, "b" -> "source1"|>,
   <|"a" -> 2, "b" -> "source2"|>,
   <|"a" -> 3, "b" -> "none"|>
   }]

Mathematica graphics

datasetSymbol = Dataset[{
   <|a -> 1, b -> "source1"|>,
   <|a -> 2, b -> "source2"|>,
   <|a -> 3, b -> "none"|>
   }]

Mathematica graphics

In order to access a column one can use

Row[{datasetString[All,Key["a"]], Spacer[20], datasetSymbol[All,Key[a]]}]

Mathematica graphics

The dataset with strings as keys can be retrieved by using either of two special shortcuts

Row[{datasetString[All,"a"], Spacer[20], datasetString[All,#a &]}]

Mathematica graphics

Similar to Associations a named column, "a", is rewritten as Key["a"].

The second shortcut is the construction #a. What appears to be the symbol a is converted to a string and becomes `Slot["a"]. This is documented in the guide/Association

Thus the Select operator can be written as

datasetString[Select[#[[Key["a"]]] < 3 &]]
atasetString[Select[#["a"] < 3 &]]
datasetString[Select[#a < 3 &]]

for the dataset with strings as keys. Note that the last form requires that the key be a string.

The Select operator for the dataset with symbols as keys can use the first two forms:

datasetSymbol[Select[#[[Key[a]]] < 3 &]]
datasetSymbol[Select[#[a] < 3 &]]

Part 2 - Type

The two datasets display differently. This is due to the Dataset type system.

Below are the FullForm of the two datasets.

FullForm[datasetString]

Mathematica graphics

FullForm[datasetSymbol]

Mathematica graphics

which accounts for why they display differently.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.