I have two datasets:

dataset1 = {{3514147200, 5.83}, {3514233600, 7.48}, {3514320000, 7.86}, {3514406400, 6.74}}
dataset2 = {{2407708800, 131.3}, {2407795200, 131.7}, {2407881600, 130.9}, {2407968000, 131}}

By down sampling to get the datasets to be the same "length", I know that there is a correlation of approximately 0.95 between the two datasets.

I have created a very large polynomial from dataset2 to approximate dataset1.

polynomial[x_] := -1.19937*10^11 - 16.2653 x - 9.10704*10^-10 x^2 + 4.74474*10^-19 x^3 + ...

When polynomial and dataset1 are plotted together, I can see that the polynomial is close to the data. I am looking for a way to measure the "distance" between the dataset and the polynomial. I tried using KolmogorovSmirnovTest but it wasn't much help, any ideas about how to do this?

  • 2
    $\begingroup$ Can you clarify what kind of data you have? "Dataset" in itself is not descriptive. $\endgroup$ – Szabolcs Jul 14 '14 at 16:39
  • 1
    $\begingroup$ Another thing you might want to clarify is what you mean by "distance". If you are asking about what is a good distance measure, it probably depends on the applications you have in mind, and it's not a question about Mathematica. stats.stackexchange.com is likely a better place to ask. If you already know what distance measure you need, then asking about how to compute it is appropriate here. $\endgroup$ – Szabolcs Jul 14 '14 at 16:47
  • 2
    $\begingroup$ So you have $(x_i,y_i)$ pairs that could be considered to approximate a function. If the domain of the two curves is the same, you could build an interpolation function from them (Interpolation) and NIntegrate the square of their difference. $\endgroup$ – Szabolcs Jul 14 '14 at 16:52
  • 2
    $\begingroup$ just fyi a high order polynomial is usually a terrible way to approximate data. You will tend to hit the points well and deviate wildly in between. $\endgroup$ – george2079 Jul 14 '14 at 18:06
  • 1
    $\begingroup$ Have you considered functions like Fit and NonlinearModelFit? $\endgroup$ – Michael E2 Jul 14 '14 at 19:37
dataset1 = Table[{x1[n], y1[n]}, {n, 5}];

poly2[x_] = a*x^2 + b*x + c;

rms1 = Norm[(poly2 /@ dataset1[[All, 1]]) - dataset1[[All, 2]]]/

rms2 = RootMeanSquare[(poly2 /@ dataset1[[All, 1]]) - 
    dataset1[[All, 2]]];

(rms1 // ComplexExpand) == rms2



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.