Converting hierarchies of rules to associations

Question

The impossibility of MatchQ-ing inside and Association MatchQ-ing Associations (MMA 10) explains a problem I have. I would like to convert hierarchical lists of rules to hierarchies of Associations. As an example:

rdata = {"a" -> {"b" -> 1, "c" -> {7, 8, 9}}, "d" -> 3}

Then

rdata /. List -> Association

returns

<|"a" -> <|"b" -> 1, "c" -> Association[7, 8, 9]|>, "d" -> 3|>

where a value that happens to be a list is converted to something meaningless, which is not wanted. I have tried a variety of things like:

rdata /. x : {__Rule} :> Association[x]

or

(rdata /. List -> Association) /. x_Association /; NotAssociationQ[x] :> List @@ x

or

rdata /. {x__Rule} :> Association[x]

but none of them solves the problem.
I finally managed it by picking out the locations of all the lists of Rules, as follows:

rtoA = (# /. {x__Rule} -> Association[x]) &
RulesToAssociation[rdata_] := Module[{pos},
  pos = Drop[#, -1] & /@ Position[rdata, Rule];
  Association @@ MapAt[rtoA, rdata, pos]]
RulesToAssociation[rdata]

which returns the desired result

<|"a" -> <|"b" -> 1, "c" -> {7, 8, 9}|>, "d" -> 3|>

In my real application the rules list comes from the Import of a fairly complicated JSON file. This solution seems to work but I'm not sure how robust it is. Does anyone have a better idea?

Answer 1

There is a ToAssociations function in the GeneralUtilities` package that is perfect for this and for converting nested JSON rules to associations:

Needs["GeneralUtilities`"]
ToAssociations@rdata
(* <|"a" -> <|"b" -> 1, "c" -> {7, 8, 9}|>, "d" -> 3|> *)

This preserves the inner most list that is not a list of rules.

---

As for your original attempts, the reason a replacement like x : {__Rule} :> Association[x] does not work is because Association is atomic and when /. replaces the outer most list with Association, you lose the ability to "inspect" inside and make replacements.

Instead, what you want is something that replaces inside out, and you can use Replace with the appropriate level spec:

Replace[rdata, x : {__Rule} :> Association[x], {0, Infinity}]
(* <|"a" -> <|"b" -> 1, "c" -> {7, 8, 9}|>, "d" -> 3|> *)

Answer 2

Version 10.2 introduces direct importing of JSON data as nested Associations.

"RawJSON":

The "RawJSON" format identifies JSON objects of the form with associations of the form in the Wolfram Language.

Among others the documentation gives this example:

Import["http://en.wikipedia.org/w/api.php?action=query&prop=links&\
titles=Wolfram%20Language&continue=&format=json", "RawJSON"]

Answer 3

There is another way to get the desired output using /. with this 2-steps approach :

rdata //. p : {__Rule} :> foo @@ p /. foo -> Association
(* <|"a" -> <|"b" -> 1, "c" -> {7, 8, 9}|>, "d" -> 3|> *)

Explanations

First step

Using //. we find at all levels in rdata the occurrences of the target structure {__Rule} that we need to replace in and we temporarily tag it as foo

fs = rdata //. p : {__Rule} :> foo @@ p

One could use a more specific rule instead, for example:

fs = rdata //. p : {Rule[_String, _] ..} :> foo @@ p

It returns a result which has already the correct final structure :

foo["a" -> foo["b" -> 1, "c" -> {7, 8, 9}], "d" -> 3]

If foo is replaced by Association then we'll get the desired output.

Second step

It appears that we can use directly /. to replace foo at all the levels and at the same time:

fs /. foo -> Association
(*<|"a" -> <|"b" -> 1, "c" -> {7, 8, 9}|>, "d" -> 3|>*)

Discussion

This result surprised me because it is in contradiction with the replacement mechanism given in @rm-rf answer :

"the reason a replacement like x : {__Rule} :> Association[x] does not work is because Association is atomic and when /. replaces the outer most list with Association, you lose the ability to "inspect" inside and make replacements"

The two reasons why the replacement in my second step should not work are : 1./ the fact that the replacement is made starting from the outermost level to the inner one and 2./ Association is Atomic

1.Difference between /. foo -> f and /. foo[x_]->f[x]

One must indeed distinguish these two cases :

foo["a", foo["b"], "c"] /. foo -> fff
(* fff["a", fff["b"], "c"] *)

and :

foo["a", foo["b"], "c"] /. foo[x__] -> fff[x]
(* fff["a", foo["b"], "c"] *)

As well explained by @WReach in the comments :

"Example 1 matches foo heads only. As they have no subparts to skip, the inner foo is visited and replaced. Example 2 matches foo[x__] which does have subparts that are skipped, and we can see the inner foo subpart is left unchanged."

which is for me more clear than the documentation of "/." (ReplaceAll) :

Clearly, the fact that I replace Heads only in my second step allow me to make this replacement at all levels.

But it should not work either because ...

2. Association is an atomic expression

which means that it is by definition "locked" in such a way that it can't be modified with tools like /..
For example :

asoc = Association[{a -> x, b -> y, c -> z}]
(* <|a -> x, b -> y, c -> z|> *)

then

asoc /. x->xx
(* <|a -> x, b -> y, c -> z|> *)

shows no replacement is possible inside the Assocation.

This means that my second step shouldn't work also :

foo["a" -> foo["b" -> 1, "c" -> {7, 8, 9}], "d" -> 3] /. foo -> Association
(* <|"a" -> <|"b" -> 1, "c" -> {7, 8, 9}|>, "d" -> 3|> *)

If the outer most foo is replaced first then it should prevent the inner replacement from taking place ... which is not the case.

This probably simply means that the whole expression to be replaced is maintained in an unevaluated form until all replacements are made...