Bug present in 10.4.1 and fixed in 11.0.1

Consider the following Dataset:

d = Dataset[<|1 -> Missing[], 2 -> 2.0|>]

Now evaluate

d[All, 1 + # &]

It gives: <|1 -> 1, 2 -> 3.|>.

Evaluate it one more time. It returns:

 <|1 -> 1 + Missing[], 2 -> 3.|>

Is this a bug?

See also the potentially related question.

  • $\begingroup$ I don't have any problems in version 10.3 on OS X 10.11.4. $\endgroup$ – xslittlegrass May 12 '16 at 2:52
  • $\begingroup$ Thanks @xslittlegrass. I am using 10.4.1 on OSX. In the comments of mathematica.stackexchange.com/questions/114562/…, another user also found the problem in 10.4.0 on windows. $\endgroup$ – ecoxlinux May 12 '16 at 3:27
  • $\begingroup$ Yes, I had confirmed the problem described on MMA 10.4.0 , Win7-64. @ecoxlinux Have you had a chance to report this to Wolfram support? $\endgroup$ – MarcoB May 12 '16 at 5:38
  • $\begingroup$ @MarcoB I did submit the report $\endgroup$ – ecoxlinux May 13 '16 at 13:50
  • $\begingroup$ If anyone can verify whether the defect exists in v10.4.0 and/or v11.0.0, please update the bug header appropriately. $\endgroup$ – WReach Jan 8 '17 at 0:17

This was a bug in the implementation of the MissingBehavior option of Dataset. The bug was present in version 10.4.1 and fixed by the time of version 11.0.1.

Analysis (current as of version 11.0.1)

By default, a dataset query defaults to the option MissingBehavior -> Automatic. This applies special behaviour that is tolerant of the presence of Missing[] in data. In the case at hand, the expression 1 + Missing[] is interpreted as simply Plus[1] (i.e. the Missing[] is interpreted as an empty sequence).

The problem arises in the implementation of this special missing behaviour. It is done by temporarily installing extra definitions upon the symbol Missing. These definitions are installed by the internal function Dataset`BlockMissingBehavior.

In version 10.4 this function would only install the extra definitions if a certain global symbol lacked an undocumented property. But there was a bug because, as the definitions were applied, the undocumented property was set upon the symbol as a side-effect. The presence of the property in the second and subsequent evaluations would prevent the extra Missing definitions from being applied and the query would behave as if the option MissingBehavior -> None had been specified.

In version 11, the special definitions are now installed unconditionally so the defect no longer appears. Versions prior to 10.4 also installed the definitions unconditionally.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.