14
$\begingroup$

Let's use the example Dataset:

dataset = Dataset[{
   <|"a" -> 1, "b" -> "x", "c" -> {1}|>,
   <|"a" -> 2, "b" -> "y", "c" -> {2, 3}|>,
   <|"a" -> 3, "b" -> "z", "c" -> {3}|>,
   <|"a" -> 4, "b" -> "x", "c" -> {4, 5}|>,
   <|"a" -> 5, "b" -> "y", "c" -> {5, 6, 7}|>,
   <|"a" -> 6, "b" -> "z", "c" -> {}|>}]

And data for a new field "d" that I'd like to add:

d = {6, 5, 4, 3, 2, 1}

I can add the field by completely unpackaging and repackaging the data:

Dataset[Map[Association,Transpose[Append[Transpose[Normal[Normal[dataset]]], Thread["d"->d]]]]]

There must be a simpler way! I would like to do this routinely and with large datasets, so I'm looking for something more compact and potentially much more efficient. What am I missing?

Improve this question
$\endgroup$
2

4 Answers 4

Reset to default
7
$\begingroup$

With this auxiliary function:

tr = Transpose[#, AllowedHeads -> All] &;

you can do

dataset[tr /* Append["d" -> {6, 5, 4, 3, 2, 1}] /*  tr]

The formatting of the result won't be as nice as the original, because of type inference limitations, but the result is correct.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Could you please show how to generalize to ds=ExampleData[{"Dataset", "Planets"}] and, say, we want to add a column with the distance from Earth ? $\endgroup$
    – b.gates.you.know.what
    Commented Nov 4, 2014 at 12:10
  • $\begingroup$ @b.gatessucks It's a bit harder. This is what I came up with (used consecutive integers for distances, to make it simple): ds[tr /* Append["distance" -> AssociationThread[Normal@ds[Keys], {1, 2, 3, 4, 5, 6, 7, 8}]] /* tr] $\endgroup$
    – Leonid Shifrin
    Commented Nov 4, 2014 at 12:21
  • $\begingroup$ Many thanks, much better than de-/re-constructing the dataset from scratch. Hopefully it won't be too long until this kind of operation is built-in. $\endgroup$
    – b.gates.you.know.what
    Commented Nov 4, 2014 at 12:51
  • 1
    $\begingroup$ @b.gatessucks Indeed, it looks like this operation must be built-in. $\endgroup$
    – Leonid Shifrin
    Commented Nov 4, 2014 at 13:18
7
$\begingroup$ 
dataset[MapThread[Append, {#, "d" -> d // Thread}] &]

or

dataset[Join[#, "d" -> d // Thread /* Map[Association], 2] &]

or

Module[{ds = Normal@dataset},
 ds[[All, "d"]] = d;
 Dataset@ds]
Improve this answer
$\endgroup$
6
$\begingroup$

Perhaps this:

Module[{i = 1}, dataset[All, <| #, "d" -> d[[i++]] |> &]]

dataset screenshot

Improve this answer
$\endgroup$
3
$\begingroup$

Either

MapIndexed[Append[#1, "d" -> d[[#2[[1]]]]] &, dataset]

or

MapIndexed[Insert[#1, "d" -> d[[#2[[1]]]], -1] &, dataset]

works

Mathematica graphics

Improve this answer
$\endgroup$
3
  • $\begingroup$ Off-beat question in the image above, what is meant by 3 levels and 27 elements? $\endgroup$
    – Doug Kimzey
    Commented Jan 14, 2015 at 13:09
  • $\begingroup$ @user1585715 3 is the max depth (that would be the depth of the list elements in c), and 27 is the number of leaves in the datastructure. $\endgroup$
    – Sjoerd C. de Vries
    Commented Jan 14, 2015 at 15:35
  • $\begingroup$ Sjoerd - Thanks very much! That makes sense. $\endgroup$
    – Doug Kimzey
    Commented Jan 14, 2015 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.