# How can I add a column into a existing Dataset?

I'm very excited about the brand new Dataset function. I have played with it in Wolfram Cloud, and haven't figured out how can I add a new column into an existing Dataset. Here is an example:

data={<|"col1"->1,"col2"->2|>,<|"col1"->3,"col2"->4|>,<|"col1"->5,"col2"->6|>};
ds=Dataset[data]


Now I can play with ds columns. For example, I can easily make calculations between columns using their names like:

ds[All, (#col1+#col2&)]


{3, 7, 11}

Another way is:

ds[All, <|"col3"-> (#col1+#col2&)|>]


<|col3->3,col3->7,col3->11|>

Now, how can I update ds, to append the brand new calculated column as "col3"? I tried:

Join[ds,ds[All, <|"col3"-> (#col1+#col2&)|>],2]


without success. It would be magic if I could just do something like:

ds[All, "col3"]=ds[All, (#col1+#col2&)]


But it does not work either.

-
It would be nice to highlight Simplex's association examples in the title/tags of this question as afaik its the only example I have found on the site of adding a "Column" to an association. –  Gordon Coale Jan 2 at 11:16
@GordonCoale nice suggestion. Done! –  Murta Jan 2 at 12:55

Here are a few ways, each of which operates upon the individual component associations.

We can explicitly construct a new association that includes all of the old columns and adds a new one:

ds[All, <| "col1"->"col1", "col2"->"col2", "col3"->(#col1 + #col2&) |>]
(* col1 col2 col3
1    2    3
3    4    7
5    6    11
*)


This has the disadvantage that we have to list all of the existing columns. To avoid this, we can use Append:

ds[All, Append[#, "col3" -> #col1 + #col2]&]
(* col1 col2 col3
1    2    3
3    4    7
5    6    11
*)


Should we wish to add multiple computed columns, we can use Join:

ds[All, # ~Join~ <| "col3" -> #col1 + #col2, "col4" -> #col1 * #col2 |> &]
(* col1 col2 col3 col4
1    2    3    2
3    4    7    12
5    6    11   30
*)

-
Cool! Tks. I'll not accept now just to respect the protocol +1 –  Murta Jun 25 '14 at 2:46
@WReach, that's a great answer. Eventually, though, you should be able to write dataset[[All, "foo"]] = {...} and have that just work (as long as the list is the right length). –  Taliesin Beynon Jul 12 '14 at 7:59

Another way that works (for one or more columns) is:

 ds[All, <|#, "col3" -> #col1 + #col2, "col4" -> #col1 - #col2|> &]


This gives:

Also, sometimes the values for the new column might not be straightforwardly computed row by row. For example, you might have calculations like this:

newcol = RotateLeft @ Normal[ds[All, (#col1 + #col2 &)]]
(* {7, 11, 3} *)
newcol2 = RandomSample[newcol]
(* {3, 11, 7} *)


In this case you can use the Association form of the Dataset. Creating a Dataset with an extra column can be done like this:

MapThread[Append, {Normal[ds], Thread["newcol" -> newcol]}] // Dataset
(* col1 col2 newcol
1    2    7
3    4    11
5    6    3
*)


One way to add multiple columns is:

Join[Normal[ds], Association /@ Thread["newcol" -> newcol],
Association /@ Thread["newcol2" -> newcol2], 2] // Dataset
(* col1 col2 newcol newcol2
1    2    7      3
3    4    11     11
5    6    3      7
*)

-
its worth noting that your very first example doesn't seem to work in 10.0.2 –  Gordon Coale Jan 8 at 23:29
@GordonCoale, that's a pity. I hope it works in 10.0.3. –  Simplex Jan 11 at 16:15