# Possible bug involving Dataset/Query and RightComposition

Bug introduced in 10.0.0 and fixed in 10.0.1

Short Version

Why are the results of the following expressions different?

Dataset[{1, 2}][f /* g, z]
Dataset[<|x->1, y->2|>][Values /* f /* g, z]

(*
g[f[{z,z}]]
g[f[z[{1,2}]]]
*)


Long Version

Consider the following dataset queries:

Dataset[{1, 2}][f , z]
Dataset[{1, 2}][f /* g, z]
Dataset[{1, 2}][f /* g /* h, z]

(*
f[{z,z}]
g[f[{z,z}]]
h[g[f[{z,z}]]]
*)


The results are as expected, with the ascending operators f, z, etc. being applied from the lowest level outward. But now consider these queries:

Dataset[<|x->1, y->2|>][Values /* f, z]
Dataset[<|x->1, y->2|>][Values /* f /* g, z]
Dataset[<|x->1, y->2|>][Values /* f /* g /*h, z]

(*
f[{z,z}]
g[f[z[{1,2}]]]
h[g[z[f[{1,2}]]]]
*)


This time, Values is used to ignore the keys in the assocation. The level 1 query operator is a composition of a descending operator with one or more ascending operators. The documentation has this to say:

When one or more descending operators are composed with one or more ascending operators (e.g. desc /* asc), the descending part will be applied, then subsequent operators will be applied to deeper levels, and lastly the ascending part will be applied to the result.

Yet note how in the second query the level two ascending operator z is no longer applied to the level two elements. Worse still, in the third query the level 2 ascending operator z is being applied after the level 1 ascending operator f.

The operator reordering is most apparent in the "compiled" forms of the queries:

DatasetCompileQuery[Values /* f, z]
DatasetCompileQuery[Values /* f /* g, z]
DatasetCompileQuery[Values /* f /* g /*h, z]

(*
DatasetWithOverrides@*GeneralUtilitiesChecked[Values/*Map[z]/*f,Identity]
DatasetWithOverrides@*GeneralUtilitiesChecked[Values/*f/*Map[z]/*g,Identity]
DatasetWithOverrides@*GeneralUtilitiesChecked[Values/*f/*g/*Map[z]/*h,Identity]
*)


Is this a bug, or a gap in my understanding?

Update

In a comment below, @TaliesinBeynon confirms that this is a bug that will be fixed in v10.0.1.

• It seems like a bug to me. As if only the last ascending operator is applied if the first of the chain is descending. If this is true, you could make any op descending/ascending with something like Select[True &] /* desc1 /*.../* descn /* Function[asc1 /* ... /* ascn@#]
– Rojo
Jul 30, 2014 at 14:45
• @Rojo this is indeed a bug. I've fixed it, hopefully should get into 10.0.1 (very close timing). Aug 11, 2014 at 22:33

Workaround

Even though I cannot explain the behaviour, I note that the level one ascending operators will remain right-composed by means of a level one subquery or by a subsequent query:

Dataset[<|x->1, y->2|>][Values /* Query[f /* g /*h], z]
Dataset[<|x->1, y->2|>][Values, z][f /* g /*h]

(*
h[g[f[{z,z}]]]
h[g[f[{z,z}]]]
*)
`
• Time to Accept this perhaps? Jan 19, 2015 at 16:56