Suppose I want to construct an association of associations, such as a list of people with attributes:

peopleFacts=<| alice-> <|age->29,shoeSize->7|>, bob-> <|age->27,sex->male|> |>

However, I want to grow and update this organically by adding facts as I learn them.

peopleFacts[["steve","hairColor"]] = "red";
peopleFacts[["bob","age"]] = "22";
peopleFacts[["steve","major"]] = "physics";

It's possible to accomplish this awkwardly by either (a) filling the database with blank entries or (b) laboriously checking at each level of association to see if an entry is blank before filling it in (except the last level, where AssociateTo helps you). But I think there must be a more elegant way. Here is what I've tried.

This method breaks because it tosses out the second key:

 In[]:= peopleFacts[["steve","hairColor"]] = "red";
        peopleFacts

Out[]:= <|steve -> red, alice-> <|age->29,shoeSize->7|>, bob-> <|age->27,sex->male|> |>

This method drops existing data:

 In[]:= peopleFacts

Out[]:= <| alice-> <|age->29,shoeSize->7|>, bob-> <|age->27,sex->male|> |>

 In[]:= AssociateTo[peopleFacts, alice-> <|"sport"->"baseball"|>;
        peopleFacts

Out[]:= <| alice-> <|sport->baseball|>, bob-> <|age->27,sex->male|> |>

This method just doesn't evaluate:

 In[]:= AssociateTo[peopleFacts[["chris"]], "favoriteFood" -> "sushi"]

Out[]:= AssociateTo[peopleFacts[["chris"]], "favoriteFood" -> "sushi"]

EDIT: Here is a way-too-awkward method adapted from this answer by SuTron.

 In[]:= peopleFacts

Out[]:= <| alice-> <|age->29,shoeSize->7|>, bob-> <|age->27,sex->male|> |>

 In[]:= Module[{temp = peopleFacts["alice"]},
          AssociateTo[temp, "sport"->"baseball"];
          AssociateTo[peopleFacts, "alice" -> temp];
        ];
        peopleFacts

Out[]:= <| alice-> <|age->29,shoeSize->7,sport->baseball|>, bob-> <|age->27,sex->male|> |>

It's not hard to imagine defining a custom update function like

  NestedAssociateTo[peopleFacts,{"steve","haircolor","red"}]

that would handle this all for you, but I'd much rather have a nice native Mathematica solution that is optimized, and that I don't have to maintain or worry about.

  • 2
    This question feels familiar but I can recall neither an answer nor the original I may be thinking of. (For the sake of my own organization alone this is vaguely related: (88442)) – Mr.Wizard Dec 20 '15 at 2:03
  • Ahh, I think my question is essentially a special case of (82915), except that additional work is needed to generalize the answer to that question for arbitrary levels of nesting. – Jess Riedel Dec 20 '15 at 14:52
  • Seeing Kuba's solution I am reminded that these are at least distantly related: (83507), (86578) – Mr.Wizard Dec 20 '15 at 16:36
  • @JessRiedel - can you edit to make your symbols vs strings consistent? You have, both bob and "bob". – alancalvitti Aug 31 '17 at 21:12
up vote 29 down vote accepted

Initial data:

peopleFacts = <|
    alice -> <|age -> 29, shoeSize -> 7|>, 
    bob -> <|age -> 27, sex -> male,  hair -> <|Color -> RGBColor[1, 0, 0]|>
    |>
|>;

Here is a version of RecurAssocMerge reduced to a single definition.

MergeNested = If[MatchQ[#, {__Association}], Merge[#, #0], Last[#]] &

MergeNested @ {peopleFacts, <|bob -> <|hair -> <|length -> 120|>|>|>}
 <|
   alice -> <|
     age -> 29, 
     shoeSize -> 7|>, 
   bob -> <|
     age -> 27, 
     sex -> male,  
     hair -> <|Color -> RGBColor[1, 0, 0], length -> 120|>
   |>
 |>

Special case of 2-level deep association

Merge[{
   peopleFacts,
   <|bob -> <|hairColor -> 1|>|>
 },
 Association
]

"Tidy" approach to write NestedMerge:

RecurAssocMerge[a : {__Association}] := Merge[a, RecurAssocMerge];
RecurAssocMerge[a_] := Last[a];
  • adding key to deep level association:

    RecurAssocMerge[
      {peopleFacts, <|bob -> <|hair -> <|length -> 120|>|>|>}
     ]
    
     <|alice -> <|age -> 29, shoeSize -> 7|>, 
       bob -> <|age -> 27, sex -> male, hair -> <|
            Color -> RGBColor[1, 0, 0], length -> 120 |>
       |>
     |>
    
  • entirely new tree

    RecurAssocMerge[
       {peopleFacts, <|kuba -> <|hair -> <|length -> 120|>|>|>}
     ]
    
     <|
        alice -> <|age -> 29, shoeSize -> 7|>, 
        bob -> <|age -> 27, sex -> male, hair -> <|Color -> RGBColor[1, 0, 0]|>
        |>, 
        kuba -> <|hair -> <|length -> 120|>|>
    |>
    

Section added by Jess Riedel:

Specialize to single new entry

RecurAssocMerge defined above is a general method for merging nested Associations. We can define an abbreviation for the special case when we are adding only a single new entry.

RecurAssocMerge[ini_Association, path_List, value_] := RecurAssocMerge[{
   ini, Fold[<|#2 -> #|> &, value, Reverse@path]
}]

Then we can just do

RecurAssocMerge[peopleFacts, {bob, hair, length}, 120]
 <|alice -> <|age -> 29, shoeSize -> 7|>, 
   bob -> <|age -> 27, sex -> male, hair -> <|
            Color -> RGBColor[1, 0, 0], length -> 120 |>
     |>
   |>

Notes

If you want to modify peopleFacts the peopleFacts = Merge... is needed of course.

  • 1
    how to modify your code if it is not known per se on what level of Association to Merge? (I mean, if bob is nested deeper?) – garej Dec 20 '15 at 11:11
  • Merge is a good solution. Would be good if it had HoldFirst as the structure could be come big. – Edmund Dec 20 '15 at 12:00
  • @Kuba, can you please modify your answer to show the general case where the Associations are nested to deeper levels? I don't understand your comment to @garej about using GroupBy. – Jess Riedel Dec 20 '15 at 15:20
  • 1
    @JessRiedel Accepted after corrections. There were some undefined functions in the last section. But I've changed it anyway to something more compact. Also, there were string keys in the last example while everything is done on symbols, so I dropped them to be consistent. I'd call the function MergeNestedAssociation but that is not important. And finally, thanks for the edit! :) – Kuba Dec 20 '15 at 19:14
  • 1
    v.useful, succinct solution of what would seem to be a common idiom (perhaps justifying a system implementation to accommodate the OP's initial tendency: peopleFacts[["bob","hair","Length"]] = "red") Related to 2249 and implements the advocated use of Part to "Creatively Set" values in List`s/Association`s (or with operator form peopleFacts @= CreativeSet[{"bob", "hair", "Length"}, "red"] where CreativeSet[pos_,value_] := Function[a,RecurAssocMerge[a, pos,value]]) – Ronald Monson Jan 20 '16 at 7:52

Update

Created an upsert function to update/insert new keys and values into a nested association structure. It automatically inserts nested associations where they do not exists and does not need to be assigned back to the original association. It updates existing keys when they are found.

ClearAll[upsert]
Attributes[upsert] = {HoldFirst};
upsert[dat_?AssociationQ, key_, value__] :=
 If[First@Dimensions@{value} == 1,
  dat[key] = value,
  (
   If[KeyExistsQ[dat, key] == False, dat[key] = <||>];
   upsert[dat[key], First@{value}, Sequence @@ Rest@{value}]
  )
  ]

Can use upsert with as many nested levels as needed.

peopleFacts = <|"alice" -> <|"age" -> 29, "shoeSize" -> 7|>, 
   "bob" -> <|"age" -> 27, "sex" -> "male"|>|>;

Insert "steve" and association "haircolor" key/value.

upsert[peopleFacts, "steve", "haircolor", "Red"];
peopleFacts

(* <|"alice" -> <|"age" -> 29, "shoeSize" -> 7|>, 
 "bob" -> <|"age" -> 27, "sex" -> "male"|>, 
 "steve" -> <|"haircolor" -> "Red"|>|> *)

Insert "tim", association "music" key/value, and nested association "rock" key/value.

upsert[peopleFacts, "tim", "music", "rock", "jimmy"];
peopleFacts

(* <|"alice" -> <|"age" -> 29, "shoeSize" -> 7|>, 
 "bob" -> <|"age" -> 27, "sex" -> "male"|>, 
 "steve" -> <|"haircolor" -> "Red"|>, 
 "tim" -> <|"music" -> <|"rock" -> "jimmy"|>|>|> *)

Update "alice" "age".

upsert[peopleFacts, "alice", "age", 25];
peopleFacts

(* <|"alice" -> <|"age" -> 25, "shoeSize" -> 7|>, 
 "bob" -> <|"age" -> 27, "sex" -> "male"|>, 
 "steve" -> <|"haircolor" -> "Red"|>, 
 "tim" -> <|"music" -> <|"rock" -> "lenny"|>|>|> *)

Original Post

Each time there is a new key that has an association as its value you must initialise it as an association. Then you can use the feature of Association that creates a key when a value is assigned to a non-existing key.

peopleFacts = <|"alice" -> <|"age" -> 29, "shoeSize" -> 7|>, "bob" -> <|"age" -> 27, "sex" -> "male"|>|>;

peopleFacts["steve"] = <||>;
peopleFacts
(* <|alice -> <|age -> 29, shoeSize -> 7|>, 
 bob -> <|age -> 27, sex -> male|>, steve -> <||>|> *)

peopleFacts["steve"]["hairColor"] = "Red";
peopleFacts
(* <|alice -> <|age -> 29, shoeSize -> 7|>, 
 bob -> <|age -> 27, sex -> male|>, steve -> <|hairColor -> Red|>|> *)

peopleFacts["bob"]["age"] = 22;
peopleFacts
(* <|alice -> <|age -> 29, shoeSize -> 7|>, 
 bob -> <|age -> 22, sex -> male|>, steve -> <|hairColor -> Red|>|> *)

peopleFacts["steve"]["major"] = "Physics";
peopleFacts
(* <|alice -> <|age -> 29, shoeSize -> 7|>, 
 bob -> <|age -> 22, sex -> male|>, 
 steve -> <|hairColor -> "Red", major -> "Physics"|>|> *)

Hope this helps.

  • I'm afraid this doesn't help much. Using your method, I have to check whether I'm writing over existing data, right? If I already have a bunch of facts about steve, then initializing peopleFacts[steve] = <||> erases them all. – Jess Riedel Dec 20 '15 at 2:29
  • When you say "Each time there is a new key that has an association as its value you must initialise it as an association", are you just unaware of a more elegant method, or do you have some reason to be sure none exists? – Jess Riedel Dec 20 '15 at 2:31
  • @JessRiedel You can use KeyExistsQ to check. – Edmund Dec 20 '15 at 2:36
  • 2
    I know, but that becomes very cumbersome if you're adding elements to a database that's several layers nested. – Jess Riedel Dec 20 '15 at 2:40
  • 1
    HoldFirst passes the first parameter by reference. This prevents a copy of it begin created (would eat up memory for large items). It also allows the association passed in to be directly updated (no need to assign back to the original association). – Edmund Dec 20 '15 at 14:15

Try this, we're using it on a daily basis

Nest[Merge,f,n]

To your starting data, slightly modified (strings vs symbols):

peopleFacts = <|"alice" -> <|"age" -> 29, "shoeSize" -> 7|>, 
   "bob" -> <|"age" -> 27, "sex" -> "male"|>|>;

And new facts:

newFacts = <|
  "steve" -> <|"hairColor" -> "red", "major" -> "physics"|>,
  "bob" -> <|"age" -> 22|>|>

Update semantics for replacing existing values: (1) add newFacts last, (2) apply Last eg:

{peopleFacts, newFacts} // Nest[Merge, Last, 2]

<|"alice" -> <|"age" -> 29, "shoeSize" -> 7|>, "bob" -> <|"age" -> 22, "sex" -> "male"|>, "steve" -> <|"hairColor" -> "red", "major" -> "physics"|>|>

Keep in mind Merge does not yet have full complement of options like JoinAcross eg inner/outer/left/right and will not impute missing keys, so often KeyIntersection is required.

Also will admit for some types of ragged hierarchies, there's no current easy way to merge all relevant branches automatically (an All level spec). In other words, knowledge of the schema is required.

  • Let us continue this discussion in chat. – Kuba Aug 4 '17 at 19:14
  • @Kuba, chat is frozen - just added a recursive approach based on your answer. Let me know if you find it useful. This is the 3rd example of query recursion that I'd like to present @ WTC. – alancalvitti Aug 31 '17 at 21:26
  • Can't focus too much on this topic atm. Does it do anything more than my MergeNested? Or is there any reason to prefer one over another? Performance? Flexibility? Because readability goes for MergeNested I guess. – Kuba Sep 1 '17 at 11:56
  • I think the way you defined MergeNested it only works on 2 levels - if so the comparison should be made to the general recursive version. – alancalvitti Sep 1 '17 at 16:27
  • I'm not sure I understand. The example is already 3 level deep: <|bob -> <|hair -> <|length -> 120|>|>|>. Could you elaborate? – Kuba Sep 1 '17 at 18:12

This recursive Query is inspired by Kuba's answer:

nestedMerge[f_] := 
  Query[Merge[Identity] /* 
    Query[All, {MatchQ[{__Association}], Identity} /* 
      Replace[{{True, data_} :> 
         nestedMerge[f][data], {False, data_} :> Query[f][data]} ]]];

Test on peopleFacts - note appended data is rewritten as nested Association vs OP:

{peopleFacts, <|steve -> <|hairColor -> red|>|>, <|
    bob -> <|age -> 22|>|>, <|steve -> <|major -> physics|>|>} // 
  nestedMerge[Last] // Dataset

enter image description here

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.