1
$\begingroup$

I have a hierarchical dataset, called finalData, like this, with 31 entries per 'outer' association. enter image description here

What I want to do is select the outer associations (let's call them 'chunks') based on calculated properties of the chunk. So, for example, I might want to keep all chunks in which the minimum #Interval value is >=900. If this was a nested list, and given that Interval is the 9th column in each subarray, I would do

Select[finalData,Min[#[[All,9]]]>=900&]

Simple! But can this be done within the Dataset paradigm, using a query based on keys? FYI I ca think of plenty of 'hybrid' methods in which one does the test calculation first using something like

finalData2[All, Min[#] >= 900 &, "Distance"]

and uses the Values[Normal[]] version of this to Pick from the Dataset, but this seems inelegant. Can it be done using a single query?

(This question is the closest but wants to filter the rows within the 'chunks', rather than the chunks themselves.)

$\endgroup$
3
$\begingroup$

Here are two approaches:

titanic = ExampleData[{"Dataset", "Titanic"}][GroupBy["class"]]

enter image description here

Since the minimal age is 1 for all classes, let's take those where the max is less than 80:

titanic[All, Max, "age"]

enter image description here

titanic[Select[Query[Max /* LessThan[80], "age"]]]

enter image description here

titanic[Select[Max@#[[All, "age"]] < 80 &]]
(* same output *)

The second example is quite straightforward, it's essentially equivalent to your example. The first one does essentially the same, but in a more dataset-like way (although one can argue whether it's actually any nicer). It works by using a subquery to select the chucks. To illustrate, consider the following partial examples:

titanic[All, Max, "age"]

titanic[All, Max /* LessThan[80], "age"]

titanic[All, Query[Max /* LessThan[80], "age"]]

titanic[Select[Query[Max /* LessThan[80], "age"]]]
$\endgroup$
1
  • $\begingroup$ Thanks so much. The second example generalizes easily to multiple tests on different keys, e.g., titanic[Select[ Max[#[[All, "age"]]] < 80 && Count[#[[All, "sex"]], "female"] > 200 &]] I can't seem to find a way to do that using the subquery format, but it's not a big deal, the other way works! $\endgroup$ – Gareth Russell May 4 at 19:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.