Bug present in 10.4.1 and fixed in 11.0.1

Consider the following Dataset:

d = Dataset[<|1 -> Missing[], 2 -> 2.0|>]

Now evaluate

d[All, 1 + # &]

It gives: <|1 -> 1, 2 -> 3.|>.

Evaluate it one more time. It returns:

 <|1 -> 1 + Missing[], 2 -> 3.|>

Is this a bug?

See also the potentially related question.

  • $\begingroup$ I don't have any problems in version 10.3 on OS X 10.11.4. $\endgroup$ Commented May 12, 2016 at 2:52
  • $\begingroup$ Thanks @xslittlegrass. I am using 10.4.1 on OSX. In the comments of mathematica.stackexchange.com/questions/114562/…, another user also found the problem in 10.4.0 on windows. $\endgroup$
    – ecoxlinux
    Commented May 12, 2016 at 3:27
  • $\begingroup$ Yes, I had confirmed the problem described on MMA 10.4.0 , Win7-64. @ecoxlinux Have you had a chance to report this to Wolfram support? $\endgroup$
    – MarcoB
    Commented May 12, 2016 at 5:38
  • $\begingroup$ @MarcoB I did submit the report $\endgroup$
    – ecoxlinux
    Commented May 13, 2016 at 13:50
  • $\begingroup$ If anyone can verify whether the defect exists in v10.4.0 and/or v11.0.0, please update the bug header appropriately. $\endgroup$
    – WReach
    Commented Jan 8, 2017 at 0:17

1 Answer 1


This was a bug in the implementation of the MissingBehavior option of Dataset. The bug was present in version 10.4.1 and fixed by the time of version 11.0.1.

Analysis (current as of version 11.0.1)

By default, a dataset query defaults to the option MissingBehavior -> Automatic. This applies special behaviour that is tolerant of the presence of Missing[] in data. In the case at hand, the expression 1 + Missing[] is interpreted as simply Plus[1] (i.e. the Missing[] is interpreted as an empty sequence).

The problem arises in the implementation of this special missing behaviour. It is done by temporarily installing extra definitions upon the symbol Missing. These definitions are installed by the internal function Dataset`BlockMissingBehavior.

In version 10.4 this function would only install the extra definitions if a certain global symbol lacked an undocumented property. But there was a bug because, as the definitions were applied, the undocumented property was set upon the symbol as a side-effect. The presence of the property in the second and subsequent evaluations would prevent the extra Missing definitions from being applied and the query would behave as if the option MissingBehavior -> None had been specified.

In version 11, the special definitions are now installed unconditionally so the defect no longer appears. Versions prior to 10.4 also installed the definitions unconditionally.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.