Problem constructing Association tables with some operators

Question

fixed in 10.0.2

---

First, set up the example Dataset from the documentation:

titanic = ExampleData[{"Dataset", "Titanic"}];

Why does this fail:

titanic[GroupBy[Key@"class"], <| "length" -> Length, "max" -> Max|>, "age"]

(* ... Part age is not applicable to {__Association}  ... *)

yet this re-expression of the query works:

titanic[GroupBy[Key@"class"], All, "age"][All, <| "length" -> Length, 
  "max" -> Max|>]

As do these modifications:

titanic[GroupBy[Key@"class"], <| "length" -> Length|>, "age"]

Strangely, Max doesn't work if substituted for Length above (bug?). Finally, this is not desirable but works without new key names:

titanic[GroupBy[Key@"class"], Query[{Length, Max}], "age"]

Answer 1

This appears to be another example of the WRI-acknowledged bug described in Possible bug involving Dataset/Query and RightComposition.

In short, the query is being "compiled" into an expression that applies the "age" operator at the wrong level.

Analysis

If we "compile" the query, we can see that the descending "age" operator is being incorrectly applied to the first level of the grouping results instead of being applied two levels below those results:

Dataset`CompileQuery @
  Query[GroupBy[Key@"class"],<|"length"->Length,"max"->Max|>,"age"]

(* ...
   GroupBy[Key[class]] /*
   Map[
     GeneralUtilities`ApplyThrough[<|length->Length,max->Max|>] @*
     GeneralUtilities`Slice[age]
   ]
   ...
*)

The fact that the Length / Max operator is being applied at the same level is also suspicious here, but it is moot given the incorrect results returned by the preceding "age" operator (preceding because of the use of @* within the Map, not /*).

Note the resemblance of the compiled query above to that of the following manifestly incorrect query:

Dataset`CompileQuery @
  Query[GroupBy[Key@"class"], "age"]

(* ...
   GroupBy[Key[class]] /*
   Map[GeneralUtilities`Slice[age]]
   ...
*)

The variation in the question that uses the simpler operator <| "length" -> Length |> does not fail outright, but it does return the incorrect value 2 for all lengths. The compiled form of this variation once again reveals the application of "age" at the wrong level:

Dataset`CompileQuery @
  Query[GroupBy[Key@"class"],<|"length"->Length|>,"age"]

(* ...
   GroupBy[Key[class]] /* 
   Map[
     GeneralUtilities`ApplyThrough[<|length->Length|>] @*
     GeneralUtilities`Slice[age]
   ]
   ...
*)

Workaround

A nested sub-query can be used as a work-around to this problem. Its compiled form applies the operators at the correct levels and produces the correct results:

Dataset`CompileQuery @
    Query[GroupBy[Key@"class"], Query@<|"length"->Length,"max"->Max|>, "age"]

(* ...
   GroupBy[Key[class]] /* 
   Map[
     Map[GeneralUtilities`Slice[age]] /*
     GeneralUtilities`ApplyThrough[<|length->Length,max->Max|>]
   ]
   ...
*)

Note how the "age" operator is now being applied at the proper level: two levels below the grouping and one level below the Length / Max application. (And also note the subtle switch from @* to /*).

Answer 2

The bug here is fairly simple: <| |> is being treated as descending, not ascending like the documentation says.

I've just fixed this, it should show up in 10.0.2.

As WReach has already pointed out, for now you can just force the <| |> operator to be ascending by wrapping it in Query:

 titanic[GroupBy[Key@"class"], Query[<|"length" -> Length, "max" -> Max|>], "age"]

Answer 3

This has been fixed in 10.0.2. On windows 7, 64 bit