2
$\begingroup$

The Dataset I am working with has the following nested structure.

assoc = 
    <|"a" -> <|"k" -> 1, "l" -> <||>|>,
      "b" -> <|"k" -> 2, "l" -> <|"x" -> <|"p" -> 7, "q" -> 8|>|>|>,
      "c" -> <|"k" -> 3, "l" -> <|"y" -> <|"p" -> 9, "q" -> 10|>, "z" -> <|"p" -> 11, "q" -> 12|>|>|>|>;

dataset = Dataset@assoc

I would like to filter out rows with some criteria on p or q while preserving the rest of the rows, columns and structure.

The following example does not preserve the structure, the k and l keys are missing.

dataset[All, "l", Select[#["p"] == 7 || #["p"] == 9 &]]

I tried this but it produces all empty rows

dataset[All, Select[#["l", "p"] == 7 || #["l", "p"] == 9 &]]

Desired result is the following Association / Dataset

assoc2 = 
    <|"a" -> <|"k" -> 1, "l" -> <||>|>,
      "b" -> <|"k" -> 2, "l" -> <|"x" -> <|"p" -> 7, "q" -> 8|>|>|>,
      "c" -> <|"k" -> 3, "l" -> <|"y" -> <|"p" -> 9, "q" -> 10|>|>|>|>;

Thanks

$\endgroup$
3
$\begingroup$

I think you can use the following syntax to do what you want:

new = dataset[All,{"l"->Select[#["p"]==7||#["p"]==9&]}]

enter image description here

And the normal form of the output:

Normal @ new

<|"a" -> <|"k" -> 1, "l" -> <||>|>, "b" -> <|"k" -> 2, "l" -> <|"x" -> <|"p" -> 7, "q" -> 8|>|>|>, "c" -> <|"k" -> 3, "l" -> <|"y" -> <|"p" -> 9, "q" -> 10|>|>|>|>

$\endgroup$
  • $\begingroup$ Thanks! I did not think of using a rule to change the values associated with the l key. Based on my very limited understanding of Association internals, all of the values are not copied, just the changes. So this should perform well in terms of memory and cpu even for large datasets. $\endgroup$ – Rohit Namjoshi Oct 9 '18 at 0:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.