# How to make Missing[] invariant for some functions without redefining them

Missing[...] "is a symbolic object, with no default evaluation rules defined."

Missing data can be tagged and enables use of filters like DeleteMissing. So it's worthwhile to normalize various types of missing values, like "" or "NA" to Missing[...].

EDIT:

Suppose a set of functions {foo,bar} are interatively applied a data column that contains Missing[] elements.

Unless the {foo,bar} are implemented to handle Missing, increasingly long unevaluated expressions bar[foo[Missing[]] are generated, which often throw Dataset Failures.

Is there a global way to make Missing[] invariant, ie f[m_Missing] --> m, for any f without actually overloading f on _Missing?

-

Is there a global way to make Missing[] invariant, ie f[m_Missing] --> m, for any f without actually overloading f on _Missing?

That is a job for UpValues, i.e. definitions made with UpSet, UpSetDelayed, TagSet, etc.:

Unprotect[Missing];

_[m_Missing] ^:= m

Protect[Missing];


Now:

foo[Missing[]]

bar[Missing["NotAvailable"]]

Missing[]

Missing["NotAvailable"]


Or:

Print /@ {1, 2, 3, Missing[], 5};


1

2

3

5

Taliesin Beynon raises an important concern in the comments: this raw method will probably break stuff. However I chose to technically answer the question as asked rather than worry about its implications.

You can restrict the pattern however you see fit, within the limitations of Mathematica, but since I saw no domain clearly delineated in the question I saw no point in proceeding.

Merely as an example we could strip only Global heads:

Unprotect[Missing];

h_[m_Missing] /; ! StringFreeQ[Context@h, "Global"] ^:= m

Protect[Missing];


Now:

{Missing[]}

Plus[1, Missing[], 2]

{Missing[]}

3 + Missing[]


Yet:

foo[Missing[]]

bar[Missing["NotAvailable"]]

Missing[]

Missing["NotAvailable"]


Again this is not intended for use; it is only an example. If you precisely define the behavior you want I can attempt to help you implement it.

-
That technique is at high risk of causing bizarre breakages, because, for example, {Missing[]} will rewrite itself as Missing[]. Missing[] is a perfectly good expression, it is context-dependent whether it should be propagated through a function. –  Taliesin Beynon Jul 17 '14 at 0:57
@TaliesinBeynon Okay, that's a good point, but isn't this what the OP requested? Since you seem to have a better understanding of the request what context do you speak of? Is there a clear pattern of which heads should and should not be stripped from Missing[]? –  Mr.Wizard Jul 17 '14 at 1:49
@MrWizard I agree you answered OP's question. I just thought that I should point out the dangers of what OP is trying to do. I can sum up my warning as follows: in a symbolic language, there is only a fuzzy line between a function and a data structure; propagating Missing might be sensible for a function, but not for a data structure. As for how to distinguish the two, I absolutely agree that it would be good to have something like $InertHeads, which includes things like List and Association. A (buggy) first stab can be found in GeneralUtilitiesPredicatesPackagePrivate$SimpleHeads –  Taliesin Beynon Jul 17 '14 at 13:39
@Mr.Wizard, it seems a reasonable approach - lacking a WRI solution that doens't require Unprotect - I'll test along w Taliesin's suggestions, possibly with an update to my Q... Congrats on the 5-digit rep, well deserved. –  alancalvitti Jul 21 '14 at 16:46
@alancalvitti I'd very much appreciate it if you collect one or two examples of Dataset being unhelpful around Failures so I can fix them :) –  Taliesin Beynon Jul 22 '14 at 5:28

Especially important are making Missing 'invariant' for things like Map and Select`, instead of having the contents of the Missing object be mapped or selected. This is a kind of monad, to those who are into that kind of thing.

We need to tackle this at a language level, perhaps with an attribute.

-