11
$\begingroup$

Since upgrading to Mathematica 10 on Mac OSX I have come across a number of instances of this error, which occurs using Probit and Logit model fit.

enter image description here

A bit of googling show this is something to do with the estimation algorithm.

But the issue is more complex. When I estimate the model straight from this dataset I get the error, but when I first take the values of the data, then fit the model, I get the expected result.

Here's an example

var = {age, gender, photo6};

mTest = SemanticImport[
   "https://dl.dropboxusercontent.com/u/3997716/test.csv"];

testFit = 
 mTest[ProbitModelFit[#, var, 
    var] &, {#age, #gender, #photo6, #rawM} &]

testFit2 = 
 ProbitModelFit[#, var, 
    var] &@(mTest[All, {"age", "gender", "photo6", "rawM"}] // 
     Normal // Values)

Output of this is

enter image description here

Why the two different results for the same calculation? Is there a workaround that allows the estimation to proceed when directly using the dataset?

$\endgroup$
  • $\begingroup$ This looks like a bug to me. $\endgroup$ – RunnyKine Aug 19 '14 at 11:40
  • $\begingroup$ I thought so too. I suspect it is a general bug in the GeneralizedLinearModelFit algorithm because I've never even seen the error before (prior to V10) and now it is popping up in a couple of other places as well (though I don't have a small working example handy). $\endgroup$ – Cameron Murray Aug 19 '14 at 11:51
  • 1
    $\begingroup$ You are using SemanticImport, which is new to V10. Could the problem lie there? How did you import your data in earlier versions of Mathematica? $\endgroup$ – m_goldberg Aug 19 '14 at 12:25
  • 4
    $\begingroup$ @m_goldberg, I don't think it has anything to do with SemanticImport. You can try importing the data with Import and create the Dataset yourself, the error still persists whereas LinearModelFit works fine. $\endgroup$ – RunnyKine Aug 19 '14 at 12:56
7
$\begingroup$

I believe that this is a bug. The rest of this response speculates as to the possible cause.

We start by observing that the test can be made to work by suppressing MissingBehaviour:

mTest[
  ProbitModelFit[#, var, var] &
, {#age, #gender, #photo6, #rawM} &
, MissingBehavior -> None
]

result screenshot

It also works if FailureAction -> None is specified instead, but then the exhibited error message about non-real values is produced (along with the correct result).

As noted elsewhere MissingBehavior is implemented by Dataset`WithOverrides. ??Dataset`WithOverrides reveals that this function temporarily alters the definitions of a number of symbols, namely those in this list:

Dataset`Overrides`PackagePrivate`$AllChangedSymbols

(* { Commonest,First,InterquartileRange,Kurtosis,Last,Mean,Median,Missing,Most,
     Quartiles,Rest,RootMeanSquare,Skewness,StandardDeviation,Total,Variance }
*)

It so happens that ProbitModelFit uses Total. We can verify that fact like this:

$data = mTest[All, {#age, #gender, #photo6, #rawM} &];

On @@ Dataset`Overrides`PackagePrivate`$AllChangedSymbols

ProbitModelFit[$data // Normal, {age, gender, photo6}, {age, gender, photo6}]

Off[]

(* ... produces many trace messages containing Total ... *)

It would appear that the patching performed by Dataset`WithOverrides is interfering with the operation of ProbitModelFit. We can simulate this by engaging in some patching of our own:

Internal`InheritedBlock[{Total}
, Unprotect @ Total
; Total[n___] /; False := Null
; ProbitModelFit[$data // Normal,{age,gender,photo6},{age,gender,photo6}]
]

result screenshot

This patch is even less invasive than the one installed by Dataset`WithOverrides, and yet it generates the same error message (and the same correct output). It would seem that ProbitModelFit is expecting Total to operate exactly as it is shipped -- nothing more, nothing less.

Conclusion

ProbitModelFit does not function properly within a query with default missing- and failure-handling. The missing-handling alters the definition of Total in a manner that causes ProbitModelFit to issue a warning message. The failure-handling sees that message and, by default, fails the whole query operation. Correct operation can be restored by either disabling the missing-handling, the failure-handling, or both.

The missing-handling is implemented by monkey-patching various low-level system components. This patching implements the proper Query semantics, at the cost of disturbing normal non-query system behaviour. Such disturbances are a frequent consequence of monkey-patching. The patching methodology explains not only the issue under discussion, but a number of other erratic Dataset behaviours logged on this site.

$\endgroup$
  • $\begingroup$ That's a brilliant answer! Your MissingBehavior workaround will suit my needs for now. $\endgroup$ – Cameron Murray Aug 20 '14 at 4:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.