2
$\begingroup$
 data = {{1, 0, 0, 0, 0, 0, 0, 60.27}, {1, 0, 0, 0, 1, 0, 0, 12.72}, {1, 0, 0, 0, 1, 0, 1, 13.5.}, {1, 1, 0, 0, 1, 0, 1, 19.77}, {1, 1, 0, 1, 1, 0, 1, 21.90}, {1, 1, 0, 1, 1, 0, 1, 22.28}}

lm = LinearModelFit[data, {x1, x2, x3, x4, x5, x6, x7}, {x1, x2, x3, x4, x5, x6, x7}]

LinearModelFit::rank: The rank of the design matrix 5 is less than the number of terms 8 in the model. The model and results based upon it may contain significant numerical error.

I only have a small data set. I receive this error when attempting to fit.

Is it possible to apply constraints in LinearModelFit? I know it is possible in NonLinearModelFit.

I want to apply the constraint x2 = x3 = x4 and x5 = x6 = x7. Is this possible? Would this remove the error message?

$\endgroup$
  • $\begingroup$ Yes. Just create your data in the form {x1,x2+x3+x4,x5+x6+x7}. But even then you have 5 parameters to estimate with just 6 sample points. $\endgroup$ – JimB Mar 20 '17 at 18:54
  • $\begingroup$ this removes the error, eliminate columns where the data has only a single value LinearModelFit[data[[All, {2, 4, 5, 7, 8}]] , {x2, x4, x5, x7}, {x2, x4, x5, x7}] You do not have enough information to say anything about the missing terms. $\endgroup$ – george2079 Mar 20 '17 at 20:49
  • $\begingroup$ my column elimination result is the same as the full result (with error message) if you insert x1->1 , by the way. $\endgroup$ – george2079 Mar 20 '17 at 21:03
  • $\begingroup$ If I change my data set to: {{25.32, 0., 0., 0., 0., 0., 0., 607}, {25.32, 0., 0., 0., 13.36, 0., 0., 1282}, {25.32, 0., 0., 0., 13.36, 0., 8.69, 1627}, {25.32, 13.21, 0., 0., 13.36, 0., 8.69, 2143}, {25, 13.21, 0., 8.94, 13.36, 0., 8.69, 2109}, {25.32, 13.21, 0., 8.94, 13.36, 0., 8.69, 2170}}, so now values not one or zero but other values. Could I apply the same method, as the two correct answers below? Thank you $\endgroup$ – SPIL Mar 21 '17 at 16:33
3
$\begingroup$

The second argument of LinearModelFit contains the regressors in the model and they are constructed as arbitrary functions of the variables in the dataset: Mathematica graphics

Thus, changing the second argument of LinearModelFit to {x1, x2 + x3 + x4, x5 + x6 + x7} (that is, defining the three regressors as $f_1 = x1$, $f_2 = x2 + x3 + x4$, and $f_3 = x5 + x6 + x7$) and using the option IncludeConstantBasis -> False (because your data already has a constant column), you can get the desired result without having to make any changes to your source data:

lm2 = LinearModelFit[data, {x1, x2 + x3 + x4, x5 + x6 + x7}, {x1, x2, x3, x4, x5, x6, x7}, 
       IncludeConstantBasis -> False];

Normal[lm2]

51.8398 x1 + 7.94734 (x2 + x3 + x4) - 22.2595 (x5 + x6 + x7)

lm2["BestFitParameters"]

{51.8398, 7.94734, -22.2595}

A simpler alternative to get the same result is to exclude x1 form the set of regressors:

lm3 = LinearModelFit[data, {x2 + x3 + x4, x5 + x6 + x7}, {x1, x2, x3, x4, x5, x6, x7}];

Normal[lm3]

51.8398 + 7.94734 (x2 + x3 + x4)- 22.2595 (x5 + x6 + x7)

lm3["BestFitParameters"]

{51.8398, 7.94734, -22.2595}

$\endgroup$
  • $\begingroup$ I can't see how you achieved to move (x2 + x3 + x4) and (x5 + x6 + x7) into the brackets? Did you miss this part of the code off? $\endgroup$ – SPIL Mar 21 '17 at 9:42
  • $\begingroup$ Thank you for adding the extra explanation and the effort you've made. It is really useful to have this method. That has clarified the situation. I think I must have misread your previous message, as I am sure I saw {x1, x2 , x3 , x4, x5 , x6 , x7}, {x1, x2, x3, x4, x5, x6, x7}, and not {x1, x2 + x3 + x4, x5 + x6 + x7}, {x1, x2, x3, x4, x5, x6, x7}. I'll go to get my eyes tested! $\endgroup$ – SPIL Mar 21 '17 at 14:30
  • $\begingroup$ If I change my data set to: {{25.32, 0., 0., 0., 0., 0., 0., 607}, {25.32, 0., 0., 0., 13.36, 0., 0., 1282}, {25.32, 0., 0., 0., 13.36, 0., 8.69, 1627}, {25.32, 13.21, 0., 0., 13.36, 0., 8.69, 2143}, {25, 13.21, 0., 8.94, 13.36, 0., 8.69, 2109}, {25.32, 13.21, 0., 8.94, 13.36, 0., 8.69, 2170}}, so now values not one or zero but other values. Could I apply the same method? $\endgroup$ – SPIL Mar 21 '17 at 16:19
  • $\begingroup$ @SPIL, thank you for the accept. Both methods should work for your new data set. $\endgroup$ – kglr Mar 21 '17 at 22:12
4
$\begingroup$

To get a fit with no warnings (which does not imply you get an adequate fit), you'll still need to remove x1 as it is a constant. Here is one such fit:

data2 = Table[{Total[data[[i, {2, 3, 4}]]], 
   Total[data[[i, {5, 6, 7}]]], data[[i, 8]]}, {i, Length[data]}]
(* {{0,0,60.27},{0,1,12.72},{0,2,13.5},{1,2,19.77},{2,2,21.9},{2,2,22.28}} *)
lm = LinearModelFit[data2, {x234, x567}, {x234, x567}];
lm["BestFitParameters"]
(* {51.839843750000014, 7.947343749999994, -22.259531250000002} *)

Update

Here is some loose algebra to show why the desired constraints can be implemented by summing some of the predictor variables to result in a simpler model where $a_2 = a_3 = a_4 = a_{234}$ and $a_5 = a_6 = a_7 = a_{567}$:

Algebra

$\endgroup$
  • $\begingroup$ Thanks Jim, that is really useful. The added algebra will come in useful too. $\endgroup$ – SPIL Mar 21 '17 at 9:46
  • $\begingroup$ If I change my data set to: {{25.32, 0., 0., 0., 0., 0., 0., 607}, {25.32, 0., 0., 0., 13.36, 0., 0., 1282}, {25.32, 0., 0., 0., 13.36, 0., 8.69, 1627}, {25.32, 13.21, 0., 0., 13.36, 0., 8.69, 2143}, {25, 13.21, 0., 8.94, 13.36, 0., 8.69, 2109}, {25.32, 13.21, 0., 8.94, 13.36, 0., 8.69, 2170}}, so now values not one or zero but other values. Could I apply the same method? $\endgroup$ – SPIL Mar 21 '17 at 16:22
  • $\begingroup$ Certainly. I recommend @kglr 's approach as you don't have to create a separate dataset. But, still, just having 6 data points and 4 parameters is expected a bit much. $\endgroup$ – JimB Mar 21 '17 at 17:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.