# Using Case to pull Keys and Values out of a nested Association

The set up:

First a helper function:

Clear[values]
values[m_, l_, v_] := <|"Math" -> m, "Log" -> l, "Value" -> v|>


I have a nested Association of the form:

RV = <|
"Top01" -> <|
"Name" -> "Top01Name",
"par01" -> values[a1,
\!$$\*SubscriptBox[\(a$$, $$"\<1\>"$$]\), 1],
"par02" -> values[b1,
\!$$\*SubscriptBox[\(b$$, $$"\<1\>"$$]\), 2],
"par03" -> values[c1,
\!$$\*SubscriptBox[\(c$$, $$"\<1\>"$$]\), 3]
|>,
"Top02" -> <|
"Name" -> "Top02Name",
"par01" -> values[aa1,
\!$$\*SubscriptBox[\(a$$, $$"\<1\>"$$]\), 1],
"par02" -> values[bb1,
\!$$\*SubscriptBox[\(b$$, $$"\<1\>"$$]\), 2],
"par03" -> values[cc1,
\!$$\*SubscriptBox[\(c$$, $$"\<1\>"$$]\), 3]
|>,
"Top3" -> <|

"SubTop01" -> <|
"Name" -> "SubTop01Name",
"par01" -> values[as1,
\!$$\*SubscriptBox[\(a$$, $$"\<1\>"$$]\), 1],
"par02" -> values[bs1,
\!$$\*SubscriptBox[\(b$$, $$"\<1\>"$$]\), 2],
"par03" -> values[cs1,
\!$$\*SubscriptBox[\(c$$, $$"\<1\>"$$]\), 3]
|>,
"SubTop02" -> <|
"Name" -> "SubTop02Name",
"par01" -> values[as2,
\!$$\*SubscriptBox[\(a$$, $$"\<1\>"$$]\), 1],
"par02" -> values[bs2,
\!$$\*SubscriptBox[\(b$$, $$"\<1\>"$$]\), 2],
"par03" -> values[cs2,
\!$$\*SubscriptBox[\(c$$, $$"\<1\>"$$]\), 3]
|>
|>
|>;


Because of the constraints on my workflow I'm guaranteed to have a unique single expression for all of the <|"Math"->#,...|> instances. (Where # is not repeated for all instances of the key "Math" in the full Association.)

One of the things that I have done successfully is make a new Association with all the "Math" symbols as the keys for the "Log" and "Value" expressions.

For example:

 LogView = Cases[RV,
KeyValuePattern[{"Math" -> a_, "Log" -> b_}] :>
Rule[a, b], \[Infinity]]


which gives:

 (* {
a1 -> \!$$\*SubscriptBox[\(a$$, $$"1"$$]\),
b1 -> \!$$\*SubscriptBox[\(b$$, $$"1"$$]\),
c1 -> \!$$\*SubscriptBox[\(c$$, $$"1"$$]\),
aa1 -> \!$$\*SubscriptBox[\(a$$, $$"1"$$]\),

}*)


and :

 NumberView = Cases[RV,
KeyValuePattern[{"Math" -> a_, "Value" -> b_}] :>
Rule[a, b], \[Infinity]]


which gives:

 (*{a1 -> 1, b1 -> 2, c1 -> 3, aa1 -> 1, bb1 -> 2, cc1 -> 3, as1 -> 1,
bs1 -> 2, cs1 -> 3, as2 -> 1, bs2 -> 2, cs2 -> 3}*)


However the last thing I need to get is a List of the Keys to a "Math" symbol and the value of the "Name" key. I have tried:

Cases[RV,  KeyValuePattern[a_ -> KeyValuePattern[
{"Name" -> b_, c_ -> KeyValuePattern["Math" -> d_]}]] :>
{d, a, b, c}, {0, \[Infinity]}]


which gives:

(*
{
{as1, "SubTop01", "SubTop01Name", "par01"},
{a1, "Top01", "Top01Name", "par01"}
}
*)


I only need the "par??" key and the parent key to the "par??" key. This key will sometimes be "Top??" or "SubTop??". Along with the value for the "Name" key which will be at the same level as the "par??" key.

The question: Where are all of the other matches?

The Backstory: I'm doing simple math with lots of parameters for many cases which requires careful bookkeeping. I also need to keep a $\LaTeX$ style log file and export my data structure in to a spreadsheet such that it can be turned into an HDF5 file. (Mathematica's Export to HDF5 is not complete so we have made a work around.)

I can now do things like this:

Get an equation for the log file

 eq = a1^2 + bs2 - (aa1 + Sqrt[cc1]);
TeXForm[eq /. LogView]


Output:

 (*
a_1^2-a_1+b_1-\sqrt{c_1}
*)


I can also do an evaluation:

  eq /. NumberView


Output:

 (*
2 - Sqrt[3]
*)


Where are all of the other matches?

They are missing because variables that appear within the KeyValuePattern will only be bound to the first key or value that meets the criterion.

Consider the following pattern:

KeyValuePattern[a_ -> KeyValuePattern[{"Name" -> b_, c_ -> KeyValuePattern["Math" -> d_]}]]


It will match an association that has any key a whose value matches the inner KeyValuePattern. a will be bound to the first such key. In the case at hand, "Top01" is that first key. Subsequent keys that also match the KeyValuePattern are not considered once the first matching key has been found. This means that "Top02" and "Top03" are never visited -- and indeed they do not appear in the result.

The same reasoning applies to the sub-patterns for "Name" and "Math". They only capture the first matching subkey in each case.

This is why the result obtained contains only two entries out of the twelve we seek.

How To Get All Matches?

To get all matches, we need to iterate across all key-value pairs of interest within any association that has been successfully matched by KeyValuePattern. It is pretty ugly:

Cases[RV
,  a:KeyValuePattern[_ -> KeyValuePattern[{"Name" -> _, _ -> KeyValuePattern["Math" -> _]}]]
:> Cases[Normal@a
, (ka_ -> b:KeyValuePattern[{"Name" -> _, _ -> KeyValuePattern["Math" -> _]}])
:> Cases[Normal@b
, (kb_ -> KeyValuePattern["Math" -> d_]) :> {d, ka, b["Name"], kb}]
]
, {0, Infinity}
] // Level[#, {-2}]&

(*
{ {as1, "SubTop01", "SubTop01Name", "par01"}
, {bs1, "SubTop01", "SubTop01Name", "par02"}
, {cs1, "SubTop01", "SubTop01Name", "par03"}
, {as2, "SubTop02", "SubTop02Name", "par01"}
, {bs2, "SubTop02", "SubTop02Name", "par02"}
, {cs2, "SubTop02", "SubTop02Name", "par03"}
, {a1, "Top01", "Top01Name", "par01"}
, {b1, "Top01", "Top01Name", "par02"}
, {c1, "Top01", "Top01Name", "par03"}
, {aa1, "Top02", "Top02Name", "par01"}
, {bb1, "Top02", "Top02Name", "par02"}
, {cc1, "Top02", "Top02Name", "par03"}
}
*)


The essential change from the original expression is that we have had to invoke Cases within each sublevel to ensure that all possible key-value pairs are considered. There is some repetition of the patterns in order to filter out unrelated key-value pairs.

Alternatives?

A slightly less verbose, but still ugly, alternative is this:

Position[RV, KeyValuePattern["Math" -> _], {0, Infinity}] //
Cases[{p___, Key[k1_], Key[k2_]}
:> {RV[[p, k1, k2, "Math"]], k1, RV[[p, k1, "Name"]], k2}]


This works by first locating the positions of all associations with a "Math" key. It then looks upward at the two containing levels to obtain the context keys. This formulation makes the assumption that "Math" associations are always contained within outer associations of the correct shape. The preceding Cases formulation does not make this assumption -- it explicitly checks to be sure. This difference in implementation might or might not be important to the application.

Both of these techniques require a lot of scanning and rescanning of the RV expression tree. For large trees, this might not perform well. The unsupported function ScanIndexed can be used with Reap to reduce the rescanning a little bit:

Needs["GeneralUtilities"]

ScanIndexed[
# /. KeyValuePattern["Math" -> d_] :>
Sow[{d, #2[[-2, 1]], Extract[RV, #2[[1;;-2]]]["Name"], #2[[-1, 1]]}] &
, RV
, -1
] // Reap // #[[2, 1]]&


... but it does not remove the rescanning completely (note Extract) and is still pretty ugly.

Mathematica really needs an improved way to access outer expression parts that contain pattern-matched elements (e.g. like the parent and ancestor axes in XPath and XQuery).

• Thank you WReach Your response is clear and works as advertised. – c186282 Oct 17 '16 at 19:54
• @WReach, the documentation seems unclear or misleading: "KeyValuePattern matches elements that appear anywhere in an association". Shouldn't it read, "...matches the first element that appears..."? – alancalvitti Jan 9 '17 at 17:44
• @alancalvitti Further down it tells us that "Every rule in Association is matched at most once" but the "first match" bit is only implicit in the example's result. I agree that an explicit statement in the Details section would be better. As would mentioning that repeating patterns won't repeat across rules. Or that specifying the patterns as an association instead of as a list changes the semantics so that order matters and repeating patterns work across rules once more. – WReach Jan 9 '17 at 19:24
• @WReach, "Every rule in Association is matched at most once" - that Association must be the argument of Replace or ReplaceAll or MatchQ (for example), not the list of rules provided in KeyValuePattern (in fact the latter are input as Rule or List, not as Association). So even if every rule in the argument Association` is matched at most once, that's inconsistent with the actual behavior that only the first matching rule is processed. – alancalvitti Jan 9 '17 at 19:29
• @alancalvitti The first example in Possible Issues states: "Every pattern in KeyvaluePattern may match only one rule." Again the "first" bit is implicit. This documentation is complicated by the ambiguity of the word "rule". Is it a rule in the assocation expression being matched, a rule to be matched within a pattern, or a rule that express a replacement? The terseness of the docs does little to resolve such ambiguity, especially since the patterns can be specified in an association (instead of a list or single rule) which changes the semantics. – WReach Jan 9 '17 at 19:55