LinearFit, are constraints possible?

Question

 data = {{1, 0, 0, 0, 0, 0, 0, 60.27}, {1, 0, 0, 0, 1, 0, 0, 12.72}, {1, 0, 0, 0, 1, 0, 1, 13.5.}, {1, 1, 0, 0, 1, 0, 1, 19.77}, {1, 1, 0, 1, 1, 0, 1, 21.90}, {1, 1, 0, 1, 1, 0, 1, 22.28}}

lm = LinearModelFit[data, {x1, x2, x3, x4, x5, x6, x7}, {x1, x2, x3, x4, x5, x6, x7}]

LinearModelFit::rank: The rank of the design matrix 5 is less than the number of terms 8 in the model. The model and results based upon it may contain significant numerical error.

I only have a small data set. I receive this error when attempting to fit.

Is it possible to apply constraints in LinearModelFit? I know it is possible in NonLinearModelFit.

I want to apply the constraint x2 = x3 = x4 and x5 = x6 = x7. Is this possible? Would this remove the error message?

Answer 1

The second argument of LinearModelFit contains the regressors in the model and they are constructed as arbitrary functions of the variables in the dataset:

Thus, changing the second argument of LinearModelFit to {x1, x2 + x3 + x4, x5 + x6 + x7} (that is, defining the three regressors as $f_1 = x1$, $f_2 = x2 + x3 + x4$, and $f_3 = x5 + x6 + x7$) and using the option IncludeConstantBasis -> False (because your data already has a constant column), you can get the desired result without having to make any changes to your source data:

lm2 = LinearModelFit[data, {x1, x2 + x3 + x4, x5 + x6 + x7}, {x1, x2, x3, x4, x5, x6, x7}, 
       IncludeConstantBasis -> False];

Normal[lm2]

51.8398 x1 + 7.94734 (x2 + x3 + x4) - 22.2595 (x5 + x6 + x7)

lm2["BestFitParameters"]

{51.8398, 7.94734, -22.2595}

A simpler alternative to get the same result is to exclude x1 form the set of regressors:

lm3 = LinearModelFit[data, {x2 + x3 + x4, x5 + x6 + x7}, {x1, x2, x3, x4, x5, x6, x7}];

Normal[lm3]

51.8398 + 7.94734 (x2 + x3 + x4)- 22.2595 (x5 + x6 + x7)

lm3["BestFitParameters"]

{51.8398, 7.94734, -22.2595}

Answer 2

To get a fit with no warnings (which does not imply you get an adequate fit), you'll still need to remove x1 as it is a constant. Here is one such fit:

data2 = Table[{Total[data[[i, {2, 3, 4}]]], 
   Total[data[[i, {5, 6, 7}]]], data[[i, 8]]}, {i, Length[data]}]
(* {{0,0,60.27},{0,1,12.72},{0,2,13.5},{1,2,19.77},{2,2,21.9},{2,2,22.28}} *)
lm = LinearModelFit[data2, {x234, x567}, {x234, x567}];
lm["BestFitParameters"]
(* {51.839843750000014, 7.947343749999994, -22.259531250000002} *)

Update

Here is some loose algebra to show why the desired constraints can be implemented by summing some of the predictor variables to result in a simpler model where $a_2 = a_3 = a_4 = a_{234}$ and $a_5 = a_6 = a_7 = a_{567}$: