All code in the question is a slight modification of code here. First I use a method george2079 used to make random points near an ellipse.
r[a_, b_, theta_, t_] :=
a Sqrt[2]/Sqrt[(1 + (a/b)^2) + (1 - (a/b)^2) Cos[2 (t - theta)]];
SeedRandom[100];
points = Table[{1.2, 1.5} + {Cos[t], Sin[t]} r[1, 0.5, \[Pi]/3, t] +
RandomVariate[NormalDistribution[0, 0.01], 2], {t,RandomVariate[UniformDistribution[{0, 2 Pi}], 100]}];
The code below is a simplified version of ubpdqn's updated approach, but it doesn't work.
lm = LinearModelFit[{#1^2, #1, #2, 2 #1 #2, #2^2} & @@@ points, {1, a, b, c, d}, {a, b, c, d}];
{p1, p2, p3, p4, p5} = lm["BestFitParameters"];
{dx, dy} = 0.5*Inverse[{{-p2, -p5}, {-p5, 1}}] . {p3, p4};
mat = {{-p2, -p5}, {-p5, 1}};
const = (-p2*dx^2 + dy^2 - 2*p5*dx*dy) - p1;
ContourPlot[-p2 (x - dx)^2 - 2*p5 (x - dx) (y - dy) + (y - dy)^2 - const == 0, {x, -1, 3}, {y, -1, 4}, Epilog -> {Point@points}]
However, the results above seem to work fairly well if I change const above to const/7.
ContourPlot[-p2 (x - dx)^2 - 2*p5 (x - dx) (y - dy) + (y - dy)^2 - const/7 == 0,
{x, 0.5, 1.9}, {y, 0.5, 2.5}, Epilog -> {Point@points}]
How do we modify the approach above so it will automatically work with any points that are approximately on an ellipse?