6
$\begingroup$

I would like to reproduce the result of multiple linear regression with Predict. In approach 1, I used matrix computation and obtained 3.2, while in approach 2, I used Predict directly and obtained 4.05.

Upon checking with other statistical software, 3.2 should be correct. May I ask why approach 2 is wrong, and how can I fix the problem?

data1 = {1, 2, 3};
data2 = {2, 3, 5};
data3 = {3, 5, 8};
data4 = {4, 5, 8.5};
X = {data1, data2, data3, data4};
Y = {1.2, 2.5, 3.3, 4.9};
ToBePredicted = {3, 6, 9};

Approach 1 - Matrix Computation - Result:3.2 (correct)

X1 = ArrayFlatten[{{X, ConstantArray[{1}, 4]}}];
X1T = Transpose[X1];
β = Inverse[X1T.X1].X1T.Y;
β.Append[ToBePredicted, 1] 

Approach 2 - Predict - Result:4.05 (wrong)

asso = MapThread[#1 -> #2 &, {X, Y}];
p = Predict[asso, Method -> "LinearRegression"];
p[ToBePredicted]
$\endgroup$
  • 2
    $\begingroup$ p2 = Predict[asso, Method -> {"LinearRegression","L2Regularization"->0}]; p2[ToBePredicted] gives the same result as β.Append[ToBePredicted, 1] $\endgroup$ – kglr May 1 '18 at 1:17
  • 1
    $\begingroup$ as stated in Documentation >> Predict >> Properties & Relations : The linear regression predictor without regularization and LinearModelFit can train equivalent models. $\endgroup$ – kglr May 1 '18 at 1:18
6
$\begingroup$

Using the suboption "L2Regularization" -> 0 for "LinearRegression" in Method option setting gives the same result as the matrix computation in OP:

p2 = Predict[asso, Method -> {"LinearRegression", "L2Regularization" -> 0}];
p2[ToBePredicted]

2.8

which is the same as

β.Append[ToBePredicted, 1]

2.8

Documentation >> Predict >> Properties & Relations :

The linear regression predictor without regularization and LinearModelFit can train equivalent models.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.