Let's take a simple case
$$y_i = \beta_0 + \beta_1 x_i$$ $$x_i \,\,\,\, y_i$$ are the observed data of independent variable and the dependent variable respectively. $$\beta_0 \,\,\,\,\beta_1 $$ are estimated by minimizing the sum of vertical distances of the observed data from the fitted line
$$s\left(\beta_0, \beta_1\right) = \Sigma_i (\epsilon_i)^2 = \Sigma_i (y_i - \beta_0 - \beta_1 x_i)^2$$ this is known as ordinary least square estimate or direct regression.
The other methods of parameter estimation consists of
(i) Inverse regression: sum of horizontal distances of the observed data from the fitted line is minimized
(ii) Major axis regression: sum of perpendicular distances of the observed data from the fitted line is minimized
(iii) Reduced major axis regression: the sum of the areas of rectangles, defined between the observed data points and the nearest point on the fitted line, is minimized.
For more details please refer to http://home.iitk.ac.in/~shalab/regression/Chapter2-Regression-SimpleLinearRegressionAnalysis.pdf page 2-4.
LinearModelFit may use ordinary least square estimate by default. Can the other three methods be performed in mathematica? If so how?

  • $\begingroup$ Of course you can develop code to implement any method you wish $\endgroup$
    – george2079
    Commented Apr 16, 2018 at 11:31
  • $\begingroup$ Is there anything readily available in mathematica? $\endgroup$
    – csk
    Commented Apr 16, 2018 at 11:33
  • $\begingroup$ Not readily available in the sense of a built in function . 2 is trivially done by reversing the data. 3 needs nminimize. $\endgroup$
    – george2079
    Commented Apr 16, 2018 at 11:45
  • $\begingroup$ ok let me have a look at nminimize. what about Reduced major axis regression, the last one I mentioned? $\endgroup$
    – csk
    Commented Apr 16, 2018 at 11:51
  • 1
    $\begingroup$ This is definitely more information than provided in your previous question: mathematica.stackexchange.com/questions/170477/…. But next time you might want to edit a closed question rather than opening a new one with the same question. $\endgroup$
    – JimB
    Commented Apr 16, 2018 at 16:00

1 Answer 1


Reduced major axis regression:

Example data:

data = Table[  {# + RandomVariate[NormalDistribution[0, .1]], 
      1 + # + RandomVariate[NormalDistribution[0, .1]]} &@ 
    RandomReal[{0, 1}] , {12}] ;

standard linear regression for comparison:

fit = LinearModelFit[data, x, x][x]

error defined as (positive) area of rectangle between point and nearest point to line:

errRMA[m_?NumericQ, b_?NumericQ, pt_List] := 
      Abs[Times @@ Subtract @@@
      Transpose[{({{1, m}, {m, m^2}}.pt - {m, -1} b)/(1 + m^2), pt}]]

note the distance formula can be found from RegionNearest[InfiniteLine[{{0,b},{1,m+b}}],{p,q}]

minimize error

rmafit = m x + b /. 
  FindMinimum[Total[errRMA[m, b  , #] & /@ data  ] , {m, b}][[2]]
Show[{ListPlot[data, AspectRatio -> 1], 
  Plot[{fit, rmafit }, {x, 0, 1}, 
   PlotStyle -> {Dashed, Black}]}]

enter image description here


nr = RegionNearest[Line[{{0, rmafit /. x -> 0}, {1, rmafit /. x -> 1}}]];
   Epilog -> ({FaceForm[None], EdgeForm[Dashed], 
      Rectangle[#, nr[#]] & /@ data})],
  Plot[rmafit , {x, 0, 1}, PlotStyle -> Black]}, AspectRatio -> 1]

enter image description here

aside, note that the reduced major axis regression error formulation has degenerate minima. If you fit a line exactly horizontal or vertical then all the rectangles have zero area, hence zero error. For robustness you probably want to do an ordinary linear regression and use those parameters as the initial guess for FindMinimum.

  • $\begingroup$ Thanks. I'll look into it in detail. $\endgroup$
    – csk
    Commented Apr 19, 2018 at 10:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.