# Bug when combining Dataset with a scoping construct and Map

The following fails in v11:

data = Dataset@<|"A" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>, "B" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>, "C" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>|>
Map[With[{num = "A"}, #[[num]]] &, data]


Returning a Failure expression about num not being a valid specification, as if it didn't get evaluated. On the other hand this works (by bypassing num):

Map[With[{num = "A"}, #[["A"]]] &, data]


I can confirm that this is a problem on at least PC and Mac. This also works fine when it's just an Association and not a Dataset.

• I don't know if this should be considered a bug or a design limitation of the overloading of Part to work in a peculiar way with Datasets. Typically one would not want the entire Function (body) to be evaluated before the actual application as that could have undesired side-effects. However without that evaluation num is not a valid specification. Consider another example without With that throws the same error: Map[(num = "A"; #[[num]]) &, data] Hopefully one of the developers see this and either explains this as a design trade-off or a bug. – Mr.Wizard Sep 3 '16 at 3:03
• @Mr.Wizard MapAt doesn't have this limitation, hence I think we have a bug here (see my answer). – Alexey Popkov Sep 3 '16 at 8:38
• @Alexey Thank you for looking deeper. I only had time for a comment before. I shall revise my conjecture: this may be part of an optimization routine for Map as applied to Dataset, analogous to but separate from compilation for packable data. In such cases Function may be processed uniquely which could explain why it appears in the simple example I gave, but not for f[d_] := (num = "A"; d[[num]]). – Mr.Wizard Sep 3 '16 at 11:59
• @Mr.Wizard Very possible, but such an optimization shouldn't significantly alter the behavior of basic functions, so it should be a bug anyway. – Alexey Popkov Sep 3 '16 at 13:43

\$Version

"11.0.0 for Microsoft Windows (64-bit) (July 28, 2016)"

data = Dataset@<|"A" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>, "B" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>, "C" -> <|"A" -> 2, "B" -> 3, "C" -> 4|>|>


Simplified example provided by Mr.Wizard produces the error:

Clear[num]
(num = "A"; #[[num]]) & /@ data
num


num


Without Map it works as expected:

Clear[num]
(num = "A"; #[[num]]) &@data
num


"A"


With usual functions and Map it is fine too:

Clear[num, f]
f[d_] := (num = "A"; d[[num]]);
f /@ data


Clear[num, f]
f[d_] := With[{num = "A"}, d[[num]]];
f /@ data


These convince me that the error is not in Part being overloaded for Dataset as suspected by Mr.Wizard.

Let us try MapAt:

Clear[num]
MapAt[(num = "A"; #[[num]]) &, data, 1]
num


"A"

Clear[num]
MapAt[(num = "A"; #[[num]]) &, data, {{1}, {2}, {3}}]


We see that MapAt works as expected (although there is a bug in typesetting - but that is a different story which is not related to MapAt).

The above examples convince me that the error is indeed a bug in Dataset when a pure function with part extraction is Mapped over it.

• But this works: (num = "A"; hello[#, num]) & /@ data) ... So Part is somehow also responsible for the problem. Or did I understand something wrong ? – SquareOne Sep 3 '16 at 23:13
• @SquareOne Yes, Part is responsible as well as Map and Function. Corrected my answer. – Alexey Popkov Sep 4 '16 at 5:22