# LinearFit, are constraints possible?

 data = {{1, 0, 0, 0, 0, 0, 0, 60.27}, {1, 0, 0, 0, 1, 0, 0, 12.72}, {1, 0, 0, 0, 1, 0, 1, 13.5.}, {1, 1, 0, 0, 1, 0, 1, 19.77}, {1, 1, 0, 1, 1, 0, 1, 21.90}, {1, 1, 0, 1, 1, 0, 1, 22.28}}

lm = LinearModelFit[data, {x1, x2, x3, x4, x5, x6, x7}, {x1, x2, x3, x4, x5, x6, x7}]


LinearModelFit::rank: The rank of the design matrix 5 is less than the number of terms 8 in the model. The model and results based upon it may contain significant numerical error.

I only have a small data set. I receive this error when attempting to fit.

Is it possible to apply constraints in LinearModelFit? I know it is possible in NonLinearModelFit.

I want to apply the constraint x2 = x3 = x4 and x5 = x6 = x7. Is this possible? Would this remove the error message?

• Yes. Just create your data in the form {x1,x2+x3+x4,x5+x6+x7}. But even then you have 5 parameters to estimate with just 6 sample points.
– JimB
Mar 20, 2017 at 18:54
• this removes the error, eliminate columns where the data has only a single value LinearModelFit[data[[All, {2, 4, 5, 7, 8}]] , {x2, x4, x5, x7}, {x2, x4, x5, x7}] You do not have enough information to say anything about the missing terms. Mar 20, 2017 at 20:49
• my column elimination result is the same as the full result (with error message) if you insert x1->1 , by the way. Mar 20, 2017 at 21:03
• If I change my data set to: {{25.32, 0., 0., 0., 0., 0., 0., 607}, {25.32, 0., 0., 0., 13.36, 0., 0., 1282}, {25.32, 0., 0., 0., 13.36, 0., 8.69, 1627}, {25.32, 13.21, 0., 0., 13.36, 0., 8.69, 2143}, {25, 13.21, 0., 8.94, 13.36, 0., 8.69, 2109}, {25.32, 13.21, 0., 8.94, 13.36, 0., 8.69, 2170}}, so now values not one or zero but other values. Could I apply the same method, as the two correct answers below? Thank you
– SPIL
Mar 21, 2017 at 16:33

The second argument of LinearModelFit contains the regressors in the model and they are constructed as arbitrary functions of the variables in the dataset:

Thus, changing the second argument of LinearModelFit to {x1, x2 + x3 + x4, x5 + x6 + x7} (that is, defining the three regressors as $f_1 = x1$, $f_2 = x2 + x3 + x4$, and $f_3 = x5 + x6 + x7$) and using the option IncludeConstantBasis -> False (because your data already has a constant column), you can get the desired result without having to make any changes to your source data:

lm2 = LinearModelFit[data, {x1, x2 + x3 + x4, x5 + x6 + x7}, {x1, x2, x3, x4, x5, x6, x7},
IncludeConstantBasis -> False];

Normal[lm2]


51.8398 x1 + 7.94734 (x2 + x3 + x4) - 22.2595 (x5 + x6 + x7)

lm2["BestFitParameters"]


{51.8398, 7.94734, -22.2595}

A simpler alternative to get the same result is to exclude x1 form the set of regressors:

lm3 = LinearModelFit[data, {x2 + x3 + x4, x5 + x6 + x7}, {x1, x2, x3, x4, x5, x6, x7}];

Normal[lm3]


51.8398 + 7.94734 (x2 + x3 + x4)- 22.2595 (x5 + x6 + x7)

lm3["BestFitParameters"]


{51.8398, 7.94734, -22.2595}

• I can't see how you achieved to move (x2 + x3 + x4) and (x5 + x6 + x7) into the brackets? Did you miss this part of the code off?
– SPIL
Mar 21, 2017 at 9:42
• Thank you for adding the extra explanation and the effort you've made. It is really useful to have this method. That has clarified the situation. I think I must have misread your previous message, as I am sure I saw {x1, x2 , x3 , x4, x5 , x6 , x7}, {x1, x2, x3, x4, x5, x6, x7}, and not {x1, x2 + x3 + x4, x5 + x6 + x7}, {x1, x2, x3, x4, x5, x6, x7}. I'll go to get my eyes tested!
– SPIL
Mar 21, 2017 at 14:30
• If I change my data set to: {{25.32, 0., 0., 0., 0., 0., 0., 607}, {25.32, 0., 0., 0., 13.36, 0., 0., 1282}, {25.32, 0., 0., 0., 13.36, 0., 8.69, 1627}, {25.32, 13.21, 0., 0., 13.36, 0., 8.69, 2143}, {25, 13.21, 0., 8.94, 13.36, 0., 8.69, 2109}, {25.32, 13.21, 0., 8.94, 13.36, 0., 8.69, 2170}}, so now values not one or zero but other values. Could I apply the same method?
– SPIL
Mar 21, 2017 at 16:19
• @SPIL, thank you for the accept. Both methods should work for your new data set.
– kglr
Mar 21, 2017 at 22:12

To get a fit with no warnings (which does not imply you get an adequate fit), you'll still need to remove x1 as it is a constant. Here is one such fit:

data2 = Table[{Total[data[[i, {2, 3, 4}]]],
Total[data[[i, {5, 6, 7}]]], data[[i, 8]]}, {i, Length[data]}]
(* {{0,0,60.27},{0,1,12.72},{0,2,13.5},{1,2,19.77},{2,2,21.9},{2,2,22.28}} *)
lm = LinearModelFit[data2, {x234, x567}, {x234, x567}];
lm["BestFitParameters"]
(* {51.839843750000014, 7.947343749999994, -22.259531250000002} *)


Update

Here is some loose algebra to show why the desired constraints can be implemented by summing some of the predictor variables to result in a simpler model where $a_2 = a_3 = a_4 = a_{234}$ and $a_5 = a_6 = a_7 = a_{567}$:

• Thanks Jim, that is really useful. The added algebra will come in useful too.
– SPIL
Mar 21, 2017 at 9:46
• If I change my data set to: {{25.32, 0., 0., 0., 0., 0., 0., 607}, {25.32, 0., 0., 0., 13.36, 0., 0., 1282}, {25.32, 0., 0., 0., 13.36, 0., 8.69, 1627}, {25.32, 13.21, 0., 0., 13.36, 0., 8.69, 2143}, {25, 13.21, 0., 8.94, 13.36, 0., 8.69, 2109}, {25.32, 13.21, 0., 8.94, 13.36, 0., 8.69, 2170}}, so now values not one or zero but other values. Could I apply the same method?
– SPIL
Mar 21, 2017 at 16:22
• Certainly. I recommend @kglr 's approach as you don't have to create a separate dataset. But, still, just having 6 data points and 4 parameters is expected a bit much.
– JimB
Mar 21, 2017 at 17:16