14
$\begingroup$

Let's use the example Dataset:

dataset = Dataset[{
   <|"a" -> 1, "b" -> "x", "c" -> {1}|>,
   <|"a" -> 2, "b" -> "y", "c" -> {2, 3}|>,
   <|"a" -> 3, "b" -> "z", "c" -> {3}|>,
   <|"a" -> 4, "b" -> "x", "c" -> {4, 5}|>,
   <|"a" -> 5, "b" -> "y", "c" -> {5, 6, 7}|>,
   <|"a" -> 6, "b" -> "z", "c" -> {}|>}]

And data for a new field "d" that I'd like to add:

d = {6, 5, 4, 3, 2, 1}

I can add the field by completely unpackaging and repackaging the data:

Dataset[Map[Association,Transpose[Append[Transpose[Normal[Normal[dataset]]], Thread["d"->d]]]]]

There must be a simpler way! I would like to do this routinely and with large datasets, so I'm looking for something more compact and potentially much more efficient. What am I missing?

$\endgroup$
6
$\begingroup$

With this auxiliary function:

tr = Transpose[#, AllowedHeads -> All] &;

you can do

dataset[tr /* Append["d" -> {6, 5, 4, 3, 2, 1}] /*  tr]

The formatting of the result won't be as nice as the original, because of type inference limitations, but the result is correct.

$\endgroup$
  • $\begingroup$ Could you please show how to generalize to ds=ExampleData[{"Dataset", "Planets"}] and, say, we want to add a column with the distance from Earth ? $\endgroup$ – b.gates.you.know.what Nov 4 '14 at 12:10
  • $\begingroup$ @b.gatessucks It's a bit harder. This is what I came up with (used consecutive integers for distances, to make it simple): ds[tr /* Append["distance" -> AssociationThread[Normal@ds[Keys], {1, 2, 3, 4, 5, 6, 7, 8}]] /* tr] $\endgroup$ – Leonid Shifrin Nov 4 '14 at 12:21
  • $\begingroup$ Many thanks, much better than de-/re-constructing the dataset from scratch. Hopefully it won't be too long until this kind of operation is built-in. $\endgroup$ – b.gates.you.know.what Nov 4 '14 at 12:51
  • 1
    $\begingroup$ @b.gatessucks Indeed, it looks like this operation must be built-in. $\endgroup$ – Leonid Shifrin Nov 4 '14 at 13:18
6
$\begingroup$
dataset[MapThread[Append, {#, "d" -> d // Thread}] &]

or

dataset[Join[#, "d" -> d // Thread /* Map[Association], 2] &]

or

Module[{ds = Normal@dataset},
 ds[[All, "d"]] = d;
 Dataset@ds]
$\endgroup$
5
$\begingroup$

Perhaps this:

Module[{i = 1}, dataset[All, <| #, "d" -> d[[i++]] |> &]]

dataset screenshot

$\endgroup$
3
$\begingroup$

Either

MapIndexed[Append[#1, "d" -> d[[#2[[1]]]]] &, dataset]

or

MapIndexed[Insert[#1, "d" -> d[[#2[[1]]]], -1] &, dataset]

works

Mathematica graphics

$\endgroup$
  • $\begingroup$ Off-beat question in the image above, what is meant by 3 levels and 27 elements? $\endgroup$ – Doug Kimzey Jan 14 '15 at 13:09
  • $\begingroup$ @user1585715 3 is the max depth (that would be the depth of the list elements in c), and 27 is the number of leaves in the datastructure. $\endgroup$ – Sjoerd C. de Vries Jan 14 '15 at 15:35
  • $\begingroup$ Sjoerd - Thanks very much! That makes sense. $\endgroup$ – Doug Kimzey Jan 14 '15 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.