# How to compute a weighted mean as a measure of central tendency?

Imagine I have data points in triplicates, as follows $$L=\begin{pmatrix} p_{11} & p_{12} & p_{13}\\ p_{21} & p_{22} & p_{23}\\ & \vdots &\\ p_{n1} & p_{n2} & p_{n3} \end{pmatrix}$$ What is the best way to filter this data, so that I remove outliers and end up with a vector of best-fit values $$\tilde{L}=(\tilde{p}_1,\tilde{p}_2,\cdots,\tilde{p}_n)$$ Thoughts: For example, if I have the plot

list = Table[
If[j == 5, {0.6, 0.7, 0.1},
RandomReal[{0. + 0.1 j, 0.3 + 0.1 j}, 3]], {j, 6}];
ListPlot[Transpose@list, PlotStyle -> Black, Frame -> True,
PlotRange -> All]


I could naturally take the mean of these values (in blue)

Show[
ListPlot[Transpose@list, PlotStyle -> Black, Frame -> True,
PlotRange -> All],
ListPlot[Transpose@{Range[6], Mean /@ list}, PlotStyle -> Blue]
]


but clearly, at $$n=5$$ I have a potential outlier, which could be removed for a better fit. What would be the ideal technique to assign a weighted mean to each triplicate based on the overall dataset, possibly based on the average standard deviation between triplicates? I have heard about Cook's distance, but I am not too sure how to apply it here.

Any ideas?

• This is a good question but it would seem to be a better question for stats.stackexchange.com. And there you'll certainly be asked about why you're tossing out data. For example, is there some reason for tossing a point (such as the thermocouple broke) rather than the point just doesn't look like it belongs.
– JimB
Commented Feb 23 at 16:36

## 1 Answer

To eliminate outlier, you may e.g. fit a linear function to your data and subtract it from the data to get the deviations. Then calculate the standard deviation: std of the deviations. Now you can reject data points that differ more than some multiple (here e.g. 2) of the std from the fit. Here is an example:

dat = Flatten[
Table[{j,
If[j == 5, {0.6, 0.7, 0.1}[[i]],
RandomReal[{0. + 0.1  j, 0.3 + 0.1  j}]]}, {j, 6}, {i, 3}], 1];
lin[x_] = Fit[dat, {1, x}, x];
std = StandardDeviation[t = dat[[All, 2]] - lin /@ dat[[All, 1]]];
dat1 = DeleteCases[dat, {x_, y_} /; Abs[y - lin[x]] > 2  std];
ListPlot[dat1, ImageSize -> {300, 200}]


• Sorry, what do you mean by "Then calculate the standard deviation: std of the deviations"? Commented Apr 15 at 15:48
• The deviations are the difference between a data point and the corresponding point on the linear fit. The deviations have a mean and a standard deviation: std. Commented Apr 15 at 15:52