14
$\begingroup$

The following fails in v11:

data = Dataset@<|"A" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>, "B" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>, "C" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>|>
Map[With[{num = "A"}, #[[num]]] &, data]

Returning a Failure expression about num not being a valid specification, as if it didn't get evaluated. On the other hand this works (by bypassing num):

Map[With[{num = "A"}, #[["A"]]] &, data]

I can confirm that this is a problem on at least PC and Mac. This also works fine when it's just an Association and not a Dataset.

$\endgroup$
  • $\begingroup$ I don't know if this should be considered a bug or a design limitation of the overloading of Part to work in a peculiar way with Datasets. Typically one would not want the entire Function (body) to be evaluated before the actual application as that could have undesired side-effects. However without that evaluation num is not a valid specification. Consider another example without With that throws the same error: Map[(num = "A"; #[[num]]) &, data] Hopefully one of the developers see this and either explains this as a design trade-off or a bug. $\endgroup$ – Mr.Wizard Sep 3 '16 at 3:03
  • $\begingroup$ @Mr.Wizard MapAt doesn't have this limitation, hence I think we have a bug here (see my answer). $\endgroup$ – Alexey Popkov Sep 3 '16 at 8:38
  • 1
    $\begingroup$ @Alexey Thank you for looking deeper. I only had time for a comment before. I shall revise my conjecture: this may be part of an optimization routine for Map as applied to Dataset, analogous to but separate from compilation for packable data. In such cases Function may be processed uniquely which could explain why it appears in the simple example I gave, but not for f[d_] := (num = "A"; d[[num]]). $\endgroup$ – Mr.Wizard Sep 3 '16 at 11:59
  • 1
    $\begingroup$ @Mr.Wizard Very possible, but such an optimization shouldn't significantly alter the behavior of basic functions, so it should be a bug anyway. $\endgroup$ – Alexey Popkov Sep 3 '16 at 13:43
10
$\begingroup$
$Version
"11.0.0 for Microsoft Windows (64-bit) (July 28, 2016)"
data = Dataset@<|"A" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>, "B" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>, "C" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>|>

dataset

Simplified example provided by Mr.Wizard produces the error:

Clear[num]
(num = "A"; #[[num]]) & /@ data
num

error message

num

Without Map it works as expected:

Clear[num]
(num = "A"; #[[num]]) &@data
num

dataset

"A"

With usual functions and Map it is fine too:

Clear[num, f]
f[d_] := (num = "A"; d[[num]]);
f /@ data

dataset

Clear[num, f]
f[d_] := With[{num = "A"}, d[[num]]];
f /@ data

dataset

These convince me that the error is not in Part being overloaded for Dataset as suspected by Mr.Wizard.

Let us try MapAt:

Clear[num]
MapAt[(num = "A"; #[[num]]) &, data, 1]
num

dataset

"A"
Clear[num]
MapAt[(num = "A"; #[[num]]) &, data, {{1}, {2}, {3}}]

dataset

We see that MapAt works as expected (although there is a bug in typesetting - but that is a different story which is not related to MapAt).

The above examples convince me that the error is indeed a bug in Dataset when a pure function with part extraction is Mapped over it.

$\endgroup$
  • $\begingroup$ But this works: (num = "A"; hello[#, num]) & /@ data) ... So Part is somehow also responsible for the problem. Or did I understand something wrong ? $\endgroup$ – SquareOne Sep 3 '16 at 23:13
  • $\begingroup$ @SquareOne Yes, Part is responsible as well as Map and Function. Corrected my answer. $\endgroup$ – Alexey Popkov Sep 4 '16 at 5:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.