6
$\begingroup$

I would like to reproduce the result of multiple linear regression with Predict. In approach 1, I used matrix computation and obtained 3.2, while in approach 2, I used Predict directly and obtained 4.05.

Upon checking with other statistical software, 3.2 should be correct. May I ask why approach 2 is wrong, and how can I fix the problem?

data1 = {1, 2, 3};
data2 = {2, 3, 5};
data3 = {3, 5, 8};
data4 = {4, 5, 8.5};
X = {data1, data2, data3, data4};
Y = {1.2, 2.5, 3.3, 4.9};
ToBePredicted = {3, 6, 9};

Approach 1 - Matrix Computation - Result:3.2 (correct)

X1 = ArrayFlatten[{{X, ConstantArray[{1}, 4]}}];
X1T = Transpose[X1];
β = Inverse[X1T.X1].X1T.Y;
β.Append[ToBePredicted, 1] 

Approach 2 - Predict - Result:4.05 (wrong)

asso = MapThread[#1 -> #2 &, {X, Y}];
p = Predict[asso, Method -> "LinearRegression"];
p[ToBePredicted]
$\endgroup$
2
  • 2
    $\begingroup$ p2 = Predict[asso, Method -> {"LinearRegression","L2Regularization"->0}]; p2[ToBePredicted] gives the same result as β.Append[ToBePredicted, 1] $\endgroup$
    – kglr
    Commented May 1, 2018 at 1:17
  • 1
    $\begingroup$ as stated in Documentation >> Predict >> Properties & Relations : The linear regression predictor without regularization and LinearModelFit can train equivalent models. $\endgroup$
    – kglr
    Commented May 1, 2018 at 1:18

1 Answer 1

6
$\begingroup$

Using the suboption "L2Regularization" -> 0 for "LinearRegression" in Method option setting gives the same result as the matrix computation in OP:

p2 = Predict[asso, Method -> {"LinearRegression", "L2Regularization" -> 0}];
p2[ToBePredicted]

2.8

which is the same as

β.Append[ToBePredicted, 1]

2.8

Documentation >> Predict >> Properties & Relations :

The linear regression predictor without regularization and LinearModelFit can train equivalent models.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.