Using Case to pull Keys and Values out of a nested Association

Question

The set up:

First a helper function:

Clear[values]
values[m_, l_, v_] := <|"Math" -> m, "Log" -> l, "Value" -> v|>

I have a nested Association of the form:

RV = <|
"Top01" -> <|
 "Name" -> "Top01Name",
 "par01" -> values[a1, 
   \!\(\*SubscriptBox[\(a\), \("\<1\>"\)]\), 1],
 "par02" -> values[b1, 
     \!\(\*SubscriptBox[\(b\), \("\<1\>"\)]\), 2],
 "par03" -> values[c1, 
    \!\(\*SubscriptBox[\(c\), \("\<1\>"\)]\), 3]
 |>,
"Top02" -> <|
 "Name" -> "Top02Name",
 "par01" -> values[aa1, 
    \!\(\*SubscriptBox[\(a\), \("\<1\>"\)]\), 1],
 "par02" -> values[bb1, 
    \!\(\*SubscriptBox[\(b\), \("\<1\>"\)]\), 2],
 "par03" -> values[cc1, 
     \!\(\*SubscriptBox[\(c\), \("\<1\>"\)]\), 3]
 |>,
"Top3" -> <|

 "SubTop01" -> <|
   "Name" -> "SubTop01Name",
   "par01" -> values[as1, 
       \!\(\*SubscriptBox[\(a\), \("\<1\>"\)]\), 1],
   "par02" -> values[bs1, 
        \!\(\*SubscriptBox[\(b\), \("\<1\>"\)]\), 2],
   "par03" -> values[cs1, 
           \!\(\*SubscriptBox[\(c\), \("\<1\>"\)]\), 3]
   |>,
 "SubTop02" -> <|
   "Name" -> "SubTop02Name",
   "par01" -> values[as2, 
           \!\(\*SubscriptBox[\(a\), \("\<1\>"\)]\), 1],
   "par02" -> values[bs2, 
          \!\(\*SubscriptBox[\(b\), \("\<1\>"\)]\), 2],
   "par03" -> values[cs2, 
           \!\(\*SubscriptBox[\(c\), \("\<1\>"\)]\), 3]
    |>
  |>
|>;

Because of the constraints on my workflow I'm guaranteed to have a unique single expression for all of the <|"Math"->#,...|> instances. (Where # is not repeated for all instances of the key "Math" in the full Association.)

One of the things that I have done successfully is make a new Association with all the "Math" symbols as the keys for the "Log" and "Value" expressions.

For example:

 LogView = Cases[RV, 
           KeyValuePattern[{"Math" -> a_, "Log" -> b_}] :> 
           Rule[a, b], \[Infinity]]

which gives:

 (* {
  a1 -> \!\(\*SubscriptBox[\(a\), \("1"\)]\), 
  b1 -> \!\(\*SubscriptBox[\(b\), \("1"\)]\), 
  c1 -> \!\(\*SubscriptBox[\(c\), \("1"\)]\),
  aa1 -> \!\(\*SubscriptBox[\(a\), \("1"\)]\), 

  }*)

and :

 NumberView = Cases[RV, 
                    KeyValuePattern[{"Math" -> a_, "Value" -> b_}] :> 
                    Rule[a, b], \[Infinity]]

which gives:

 (*{a1 -> 1, b1 -> 2, c1 -> 3, aa1 -> 1, bb1 -> 2, cc1 -> 3, as1 -> 1, 
   bs1 -> 2, cs1 -> 3, as2 -> 1, bs2 -> 2, cs2 -> 3}*)

However the last thing I need to get is a List of the Keys to a "Math" symbol and the value of the "Name" key. I have tried:

Cases[RV,  KeyValuePattern[a_ -> KeyValuePattern[
   {"Name" -> b_, c_ -> KeyValuePattern["Math" -> d_]}]] :> 
{d, a, b, c}, {0, \[Infinity]}]

which gives:

(*
{
 {as1, "SubTop01", "SubTop01Name", "par01"}, 
 {a1, "Top01", "Top01Name", "par01"}
}
*)

I only need the "par??" key and the parent key to the "par??" key. This key will sometimes be "Top??" or "SubTop??". Along with the value for the "Name" key which will be at the same level as the "par??" key.

The question: Where are all of the other matches?

The Backstory: I'm doing simple math with lots of parameters for many cases which requires careful bookkeeping. I also need to keep a $\LaTeX$ style log file and export my data structure in to a spreadsheet such that it can be turned into an HDF5 file. (Mathematica's Export to HDF5 is not complete so we have made a work around.)

I can now do things like this:

Get an equation for the log file

 eq = a1^2 + bs2 - (aa1 + Sqrt[cc1]);
 TeXForm[eq /. LogView]

Output:

 (*
  a_1^2-a_1+b_1-\sqrt{c_1}
 *)

I can also do an evaluation:

  eq /. NumberView

Output:

 (*
 2 - Sqrt[3]
 *)

Answer 1

Where are all of the other matches?

They are missing because variables that appear within the KeyValuePattern will only be bound to the first key or value that meets the criterion.

Consider the following pattern:

KeyValuePattern[a_ -> KeyValuePattern[{"Name" -> b_, c_ -> KeyValuePattern["Math" -> d_]}]]

It will match an association that has any key a whose value matches the inner KeyValuePattern. a will be bound to the first such key. In the case at hand, "Top01" is that first key. Subsequent keys that also match the KeyValuePattern are not considered once the first matching key has been found. This means that "Top02" and "Top03" are never visited -- and indeed they do not appear in the result.

The same reasoning applies to the sub-patterns for "Name" and "Math". They only capture the first matching subkey in each case.

This is why the result obtained contains only two entries out of the twelve we seek.

How To Get All Matches?

To get all matches, we need to iterate across all key-value pairs of interest within any association that has been successfully matched by KeyValuePattern. It is pretty ugly:

Cases[RV
,  a:KeyValuePattern[_ -> KeyValuePattern[{"Name" -> _, _ -> KeyValuePattern["Math" -> _]}]]
     :> Cases[Normal@a
        , (ka_ -> b:KeyValuePattern[{"Name" -> _, _ -> KeyValuePattern["Math" -> _]}])
            :> Cases[Normal@b
              , (kb_ -> KeyValuePattern["Math" -> d_]) :> {d, ka, b["Name"], kb}]
        ]
, {0, Infinity}
] // Level[#, {-2}]&

(*
{ {as1, "SubTop01", "SubTop01Name", "par01"}
, {bs1, "SubTop01", "SubTop01Name", "par02"}
, {cs1, "SubTop01", "SubTop01Name", "par03"}
, {as2, "SubTop02", "SubTop02Name", "par01"}
, {bs2, "SubTop02", "SubTop02Name", "par02"}
, {cs2, "SubTop02", "SubTop02Name", "par03"}
, {a1, "Top01", "Top01Name", "par01"}
, {b1, "Top01", "Top01Name", "par02"}
, {c1, "Top01", "Top01Name", "par03"}
, {aa1, "Top02", "Top02Name", "par01"}
, {bb1, "Top02", "Top02Name", "par02"}
, {cc1, "Top02", "Top02Name", "par03"}
}
*)

The essential change from the original expression is that we have had to invoke Cases within each sublevel to ensure that all possible key-value pairs are considered. There is some repetition of the patterns in order to filter out unrelated key-value pairs.

Alternatives?

A slightly less verbose, but still ugly, alternative is this:

Position[RV, KeyValuePattern["Math" -> _], {0, Infinity}] //
 Cases[{p___, Key[k1_], Key[k2_]}
         :> {RV[[p, k1, k2, "Math"]], k1, RV[[p, k1, "Name"]], k2}]

This works by first locating the positions of all associations with a "Math" key. It then looks upward at the two containing levels to obtain the context keys. This formulation makes the assumption that "Math" associations are always contained within outer associations of the correct shape. The preceding Cases formulation does not make this assumption -- it explicitly checks to be sure. This difference in implementation might or might not be important to the application.

Both of these techniques require a lot of scanning and rescanning of the RV expression tree. For large trees, this might not perform well. The unsupported function ScanIndexed can be used with Reap to reduce the rescanning a little bit:

Needs["GeneralUtilities`"]

ScanIndexed[
  # /. KeyValuePattern["Math" -> d_] :>
      Sow[{d, #2[[-2, 1]], Extract[RV, #2[[1;;-2]]]["Name"], #2[[-1, 1]]}] &
, RV
, -1
] // Reap // #[[2, 1]]&

... but it does not remove the rescanning completely (note Extract) and is still pretty ugly.

Mathematica really needs an improved way to access outer expression parts that contain pattern-matched elements (e.g. like the parent and ancestor axes in XPath and XQuery).