11
$\begingroup$

I am wondering if there is a straightforward way to add a new column to a dataset if the column needs to be computed from the key values rather than other columns. Here is a simple example.

dataset=<|101-><|"Column1"->"a"|>,
          102-><|"Column1"->"b"|>,
          201-><|"Column1"->"c"|>|>//Dataset

I want to add Column2 = f[Key], where Key is {101,102,201} and, say,

f[x_]:=FromDigits@Take[IntegerDigits[x], 1]

So far it seems I have to go a long way around: extract the key column, apply the function to it, then somehow merge the result with the dataset, which itself needs to be converted to an association and then back. Is there maybe a better way?

P.S. I have a unique row key due to an earlier reshape operation, which I could only achieve using GroupBy, which automatically created a key. It feels to me the functionality with unique row keys is somewhat raw. On one hand, they are supported by some dataset related functions like GroupBy and are shown nicely in the output. On the other hand, joining two dataset on external key from one and a unique key from the other is probably going to be a hassle.

$\endgroup$
11
$\begingroup$

After some fiddling, I have discovered that MapIndexed works for associations and came up with this solution:

dataset[MapIndexed[Append[#1, "Column2" -> f@#2[[1, 1]]] &]]

or, more cryptically,

dataset[MapIndexed[<|#1, "Column2" -> f@#2[[1, 1]]|> &]]
$\endgroup$
  • $\begingroup$ Nice, didn't know it works with associations $\endgroup$ – Kuba Aug 26 '16 at 14:19
5
$\begingroup$

I think Andrei's response using MapIndexed is superior, but I'll offer up AssociationMap as an alternative -- partly because it is a legitimate contender and partly to illustrate a work-around for a common Dataset problem.

First, here is AssociationMap in action:

dataset[AssociationMap[#[[1]] -> <| #[[2]], "Column2" -> f[#[[1]]] |> &]]

dataset screenshot

The result is structurally correct, but unfortunately the Dataset type inferencer is presently incapable of determining the resulting type correctly (as of V11). As a consequence, it visualizes the dataset in a less than useful fashion.

We must invoke the type deducer to fix up the visual rendition. We do this by adding a terminal Dataset operator to the query:

dataset[AssociationMap[#[[1]] -> <|#[[2]], "Column2" -> f[#[[1]]]|> &] /* Dataset]

dataset screenshot

$\endgroup$
1
$\begingroup$

Another alternative:

Association /@ KeyValueMap[# -> <|#2, "Column2" -> f[#]|> &, dataset]

or

Map[Association] @ KeyValueMap[# -> <|#2, "Column2" -> f[#]|> &] @ dataset
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.