14
$\begingroup$

Let's use the example Dataset:

dataset = Dataset[{
   <|"a" -> 1, "b" -> "x", "c" -> {1}|>,
   <|"a" -> 2, "b" -> "y", "c" -> {2, 3}|>,
   <|"a" -> 3, "b" -> "z", "c" -> {3}|>,
   <|"a" -> 4, "b" -> "x", "c" -> {4, 5}|>,
   <|"a" -> 5, "b" -> "y", "c" -> {5, 6, 7}|>,
   <|"a" -> 6, "b" -> "z", "c" -> {}|>}]

And data for a new field "d" that I'd like to add:

d = {6, 5, 4, 3, 2, 1}

I can add the field by completely unpackaging and repackaging the data:

Dataset[Map[Association,Transpose[Append[Transpose[Normal[Normal[dataset]]], Thread["d"->d]]]]]

There must be a simpler way! I would like to do this routinely and with large datasets, so I'm looking for something more compact and potentially much more efficient. What am I missing?

$\endgroup$
2

4 Answers 4

7
$\begingroup$

With this auxiliary function:

tr = Transpose[#, AllowedHeads -> All] &;

you can do

dataset[tr /* Append["d" -> {6, 5, 4, 3, 2, 1}] /*  tr]

The formatting of the result won't be as nice as the original, because of type inference limitations, but the result is correct.

$\endgroup$
4
  • $\begingroup$ Could you please show how to generalize to ds=ExampleData[{"Dataset", "Planets"}] and, say, we want to add a column with the distance from Earth ? $\endgroup$ Nov 4, 2014 at 12:10
  • $\begingroup$ @b.gatessucks It's a bit harder. This is what I came up with (used consecutive integers for distances, to make it simple): ds[tr /* Append["distance" -> AssociationThread[Normal@ds[Keys], {1, 2, 3, 4, 5, 6, 7, 8}]] /* tr] $\endgroup$ Nov 4, 2014 at 12:21
  • $\begingroup$ Many thanks, much better than de-/re-constructing the dataset from scratch. Hopefully it won't be too long until this kind of operation is built-in. $\endgroup$ Nov 4, 2014 at 12:51
  • 1
    $\begingroup$ @b.gatessucks Indeed, it looks like this operation must be built-in. $\endgroup$ Nov 4, 2014 at 13:18
7
$\begingroup$
dataset[MapThread[Append, {#, "d" -> d // Thread}] &]

or

dataset[Join[#, "d" -> d // Thread /* Map[Association], 2] &]

or

Module[{ds = Normal@dataset},
 ds[[All, "d"]] = d;
 Dataset@ds]
$\endgroup$
6
$\begingroup$

Perhaps this:

Module[{i = 1}, dataset[All, <| #, "d" -> d[[i++]] |> &]]

dataset screenshot

$\endgroup$
3
$\begingroup$

Either

MapIndexed[Append[#1, "d" -> d[[#2[[1]]]]] &, dataset]

or

MapIndexed[Insert[#1, "d" -> d[[#2[[1]]]], -1] &, dataset]

works

Mathematica graphics

$\endgroup$
3
  • $\begingroup$ Off-beat question in the image above, what is meant by 3 levels and 27 elements? $\endgroup$ Jan 14, 2015 at 13:09
  • $\begingroup$ @user1585715 3 is the max depth (that would be the depth of the list elements in c), and 27 is the number of leaves in the datastructure. $\endgroup$ Jan 14, 2015 at 15:35
  • $\begingroup$ Sjoerd - Thanks very much! That makes sense. $\endgroup$ Jan 14, 2015 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.