I have a data set where there is a lot of missing values; they are labelled Missing["Non-mesuré"]
. When I append this data to another list of same length which has no missing data, and try to fit a line to those two lists with LinearModelFit[]
, I get the following error message:
LinearModelFit::notdata: The first argument is not a vector, matrix, or a list containing a design matrix and response vector.
I get that Missing["Non-mesuré"]
is not data, but isn't the point of Missing[]
to allow for missing values?
Here is a sample of the data:
data={{2.2,Missing[Non-mesuré]},{1.8,Missing[Non-mesuré]},{1.8,Missing[Non-mesuré]},{1.7,Missing[Non-mesuré]},{2.1,Missing[Non-mesuré]},{2.1,Missing[Non-mesuré]},{1.8,Missing[Non-mesuré]},{2.2,Missing[Non-mesuré]},{2.2,Missing[Non-mesuré]},{2.2,Missing[Non-mesuré]},{2.5,Missing[Non-mesuré]},{2.1,Missing[Non-mesuré]},{2.1,Missing[Non-mesuré]},{2.1,Missing[Non-mesuré]},{2.3,Missing[Non-mesuré]},{2.3,Missing[Non-mesuré]},{1.7,Missing[Non-mesuré]},{2.1,Missing[Non-mesuré]},{1.7,Missing[Non-mesuré]},{1.8,Missing[Non-mesuré]},{2.1,Missing[Non-mesuré]},{2.1,Missing[Non-mesuré]},{1.5,0},{1.5,0},{1.7,0},{1.1,Missing[Non-mesuré]},{3,0},{1.6,0},{2.1,0},{1.7,0},{1.8,0},{1.7,0},{1.6,Missing[Non-mesuré]},{1.5,0},{2.2,0},{2.3,0},{2.3,0},{1.9,0},{1.9,0},{1.9,0},{1.3,0},{1.9,0},{1.9,0},{1.9,0},{1.9,0},{1.9,0},{1.3,0},{1.9,0},{1.9,0},{2,0},{2,0},{2,0},{2,0},{1,0},{1.4,0},{1.9,0},{1.8,0},{1.7,0},{1.5,0},{2,0},{2,0.333333},{2,0.25},{2.2,0.0208333},{1.7,0},{1.7,0},{1.7,0.0416667},{1.7,0},{2,0},{2,0},{1.7,0},{2.3,0},{1.7,0},{1.7,0.0138889},{1.7,0},{1.7,0.0208333},{2,0},{1.7,0},{2.4,0},{1.7,0},{0.5,0},{2,0},{1.7,0},{1.7,0.0416667},{1.7,0.0138889},{1.7,0.0208333},{2.1,0},{1.7,0},{2.1,0.0104167},{3.4,0.0208333}}
And here is the code I first tried:
LinearModelFit[data,x,x]
I also tried to remove the data points that contained missing values, without success:
LinearModelFit[If[#[[2]]==Missing["Non-mesuré"],Nothing,#]&/@data,x,x]
So either it is possible to make the fit work with the missing data, or there is a way to exclude it when doing the fit. Either solution would brighten my day.
DeleteCases[data,{_Missing,_}|{_,_Missing}]
$\endgroup$ – Julien Kluge Dec 12 '16 at 16:40Missing
seems to be mostly for use inAssociation
s, for returning missing values from Wolfram's curated data likeCountryData
orElementData
. It seems to be compatible with visualization tools likeListLinePlot
. Otherwise, I suspect it's not useful. It's certainly unlikely to be useful for numerical functions and calculations. $\endgroup$ – march Dec 12 '16 at 16:51LinearModelFit[Select[data, #[[2]] != "Missing[Non-mesuré]" &], x, x]
. $\endgroup$ – JimB Dec 12 '16 at 18:10Cases[data, {_?NumericQ ..}]
. This will also stripIndeterminate
, undefined symbols and so on. $\endgroup$ – george2079 Dec 12 '16 at 18:13