16 added 157 characters in body edited Jan 26 '15 at 21:21 chris 12.7k44 gold badges4545 silver badges121121 bronze badges In that case, you can either use a parametric fit (as suggested by @stevenvh) or a non-parametric fit, say, using splinesSplines (or any parametrisation for which you can weight the relative degree of smoothness of your fit). Let me illustrate this, starting with your dataset: Now the original solution iswas to use (Hermite) interpolation; but letslet's also define a Spline interpolation we can visually see that the grey curve (the splineSpline) provides a better representation of the derivative at the origin. Indeed is closer to the underlying value of -0.0037-0.0037, than the value returned by the original method (red dashed curve): A cubic B-splineSpline curve with 2525 control points seems reasonable here for fitting that particular curve: ShowLet me show the data with the curve: The main virtue of this second example is that we now have a control parameter, namely the number of control points, which we can adapt to the level of noise in order to get an accurate estimate of the derivative. More generally, fitting with a variable amount of smoothing can be formalized in the context of Maximum a Posteriori analysisMaximum a Posteriori analysis. There are even methods (such as cross validation) to estimate automatically what the proper amount of smoothing should be. As an alternative to using the number of splinesSplines basis to control the smoothness of the curve, let us consider an explicit penalty function. The idea here is that instead of solving for the best (spline) weights in the least square sense (a maximum likelihood solution solution while assuming Gaussian statistics), we find these, subject to a prior corresponding to a 'roughness' penalty (which allows us to tune how smooth the spine function should be, which involves adding a tunable cost to unsmooth spline). In that case you can either use a parametric fit (as suggested by @stevenvh) or a non-parametric fit, say, using splines (or any parametrisation for which you can weight the relative degree of smoothness of your fit). Let me illustrate this, starting with your dataset: Now the original solution is to use (Hermite) interpolation; but lets also define a Spline interpolation we can visually see that the grey curve (the spline) provides a better representation of the derivative at the origin. Indeed is closer to the underlying value of -0.0037, than the value returned by the original method (red dashed curve) A cubic B-spline curve with 25 control points seems reasonable here for fitting that particular curve: Show the data with the curve: The main virtue of this second example is that we now have a control parameter, namely the number of control points, which we can adapt to the level of noise in order to get an accurate estimate of the derivative. More generally, fitting with a variable amount of smoothing can be formalized in the context of Maximum a Posteriori analysis. There are even methods (such as cross validation) to estimate automatically what the proper amount of smoothing should be. As an alternative to using the number of splines basis to control the smoothness of the curve, let us consider an explicit penalty function. The idea here is that instead of solving for the best (spline) weights in the least square sense (a maximum likelihood solution solution while assuming Gaussian statistics), we find these, subject to a prior corresponding to a 'roughness' penalty (which allows us to tune how smooth the spine function should be, which involves adding a tunable cost to unsmooth spline). In that case, you can either use a parametric fit (as suggested by @stevenvh) or a non-parametric fit, say, using Splines (or any parametrisation for which you can weight the relative degree of smoothness of your fit). Let me illustrate this, starting with your dataset: Now the original solution was to use (Hermite) interpolation; but let's also define a Spline interpolation we can visually see that the grey curve (the Spline) provides a better representation of the derivative at the origin. Indeed is closer to the underlying value of -0.0037, than the value returned by the original method (red dashed curve): A cubic B-Spline curve with 25 control points seems reasonable here for fitting that particular curve: Let me show the data with the curve: The main virtue of this second example is that we now have a control parameter, namely the number of control points, which we can adapt to the level of noise in order to get an accurate estimate of the derivative. More generally, fitting with a variable amount of smoothing can be formalized in the context of Maximum a Posteriori analysis. There are even methods (such as cross validation) to estimate automatically what the proper amount of smoothing should be. As an alternative to using the number of Splines basis to control the smoothness of the curve, let us consider an explicit penalty function. The idea here is that instead of solving for the best (spline) weights in the least square sense (a maximum likelihood solution solution while assuming Gaussian statistics), we find these, subject to a prior corresponding to a 'roughness' penalty (which allows us to tune how smooth the spine function should be, which involves adding a tunable cost to unsmooth spline). 15 added 98 characters in body edited Jun 8 '13 at 10:26 chris 12.7k44 gold badges4545 silver badges121121 bronze badges In that case you can either use a parametric fit (as suggested by @stevenvh) or a non parametric-parametric fit, say, using splines (or any parametrisation for which you can weight the relative degree of smoothness of your fit). Let me illustrate this, starting with your dataset.: LetsLet us define a noisy set: Butbut lets also define a Spline interpolation is closer to the underlying value of -0.0037, than the value returned by the original method (red dashed curve) f0' (* 0.0120881 *).  Of course, statistically they both yield something reasonable BSplineFunction[ctrlpts, SplineDegree -> 3]' // #[]/#[] & (* 39.7595 *),  which, given the amount of noise, is a reasonable estimate. On average, the non parametric-parametric estimate is fine: The main virtue of this second example is that now we now have a control parameter, namely the number of control points, which we can adapt to the level of noise in order to get an accurate estimate of the derivative. More generally, fitting with a variable amount of smoothing can be formalized in the context of Maximum a Posteriori analysis. There are even methods (such as cross validation) to estimate automatically what the proper amount of smoothing should be. asAs an alternative to using the number of splines basis to control the smoothness of the curve, let us consider an explicit penalty function. The idea here is that instead of solving for the best (spline) weights in the least square sense (a maximum likelihood solution solution while assuming Gaussian statistics), we find these, subject to a prior corresponding to a roughness'roughness' penalty (which allows us to tune how smooth the spine function should be, which involves adding a tunable cost to unsmooth spline). LetsLet us start with an undersmoothunder-smooth solution (lambda=-4.25) betterBetter? As far as the estimated derivative is concerned, we can define with mathematica the derivative of our spline basis In closing, Wikipedia page on this topic is rather good. In that case you can either use a parametric fit (as suggested by @stevenvh) or a non parametric fit, say using splines (or any parametrisation for which you can weight the relative degree of smoothness of your fit). Let me illustrate this, starting with your dataset. Lets define a noisy set: But lets also define a Spline interpolation is closer to the underlying value of -0.0037 than the value returned by the original method (red dashed curve) f0' (* 0.0120881 *)  Of course statistically they both yield something reasonable BSplineFunction[ctrlpts, SplineDegree -> 3]' // #[]/#[] & (* 39.7595 *)  which, given the amount of noise, is a reasonable estimate. On average the non parametric estimate is fine: The main virtue of this second example is that now we have a control parameter, namely the number of control points which we can adapt to the level of noise in order to get an accurate estimate of the derivative. More generally, fitting with variable amount of smoothing can be formalized in the context of Maximum a Posteriori analysis. There are even methods (such as cross validation) to estimate automatically what the proper amount of smoothing should be. as an alternative to using the number of splines basis to control the smoothness of the curve, let us consider an explicit penalty function. The idea here is that instead of solving for the best (spline) weights in the least square sense (a maximum likelihood solution solution while assuming Gaussian statistics), we find these, subject to a prior corresponding to a roughness penalty (which allows us to tune how smooth the spine function should be, which involves adding a tunable cost to unsmooth spline). Lets start with an undersmooth solution (lambda=-4.25) better? As far as the estimated derivative is concerned, we can define the derivative of our spline basis In that case you can either use a parametric fit (as suggested by @stevenvh) or a non-parametric fit, say, using splines (or any parametrisation for which you can weight the relative degree of smoothness of your fit). Let me illustrate this, starting with your dataset: Let us define a noisy set: but lets also define a Spline interpolation is closer to the underlying value of -0.0037, than the value returned by the original method (red dashed curve) f0' (* 0.0120881 *).  Of course, statistically they both yield something reasonable BSplineFunction[ctrlpts, SplineDegree -> 3]' // #[]/#[] & (* 39.7595 *),  which, given the amount of noise, is a reasonable estimate. On average, the non-parametric estimate is fine: The main virtue of this second example is that we now have a control parameter, namely the number of control points, which we can adapt to the level of noise in order to get an accurate estimate of the derivative. More generally, fitting with a variable amount of smoothing can be formalized in the context of Maximum a Posteriori analysis. There are even methods (such as cross validation) to estimate automatically what the proper amount of smoothing should be. As an alternative to using the number of splines basis to control the smoothness of the curve, let us consider an explicit penalty function. The idea here is that instead of solving for the best (spline) weights in the least square sense (a maximum likelihood solution solution while assuming Gaussian statistics), we find these, subject to a prior corresponding to a 'roughness' penalty (which allows us to tune how smooth the spine function should be, which involves adding a tunable cost to unsmooth spline). Let us start with an under-smooth solution (lambda=-4.25) Better? As far as the estimated derivative is concerned, we can define with mathematica the derivative of our spline basis In closing, Wikipedia page on this topic is rather good. 14 added 1 characters in body edited Sep 24 '12 at 14:22 chris 12.7k44 gold badges4545 silver badges121121 bronze badges  df[x_] = BSplineFunction[ctrlpts[0.25], SplineDegree -> 3]'[x]3]''[x]   df[x_] = BSplineFunction[ctrlpts[0.25], SplineDegree -> 3]'[x]   df[x_] = BSplineFunction[ctrlpts[0.25], SplineDegree -> 3]''[x]  13 added 127 characters in body edited Sep 24 '12 at 13:45 chris 12.7k44 gold badges4545 silver badges121121 bronze badges 12 added 8 characters in body edited Sep 24 '12 at 13:01 chris 12.7k44 gold badges4545 silver badges121121 bronze badges Mod Removes Wiki by J. M. will be back soon♦ occurred Sep 24 '12 at 10:16 11 added explicit penalty edited Sep 24 '12 at 10:02 chris 12.7k44 gold badges4545 silver badges121121 bronze badges 10 added explicit penalty edited Sep 24 '12 at 9:55 chris 12.7k44 gold badges4545 silver badges121121 bronze badges 9 added 66 characters in body edited Sep 23 '12 at 18:07 chris 12.7k44 gold badges4545 silver badges121121 bronze badges 8 added 66 characters in body edited Sep 23 '12 at 18:02 chris 12.7k44 gold badges4545 silver badges121121 bronze badges 7 added 394 characters in body edited Sep 23 '12 at 17:08 chris 12.7k44 gold badges4545 silver badges121121 bronze badges 6 added 215 characters in body edited Sep 23 '12 at 16:19 chris 12.7k44 gold badges4545 silver badges121121 bronze badges 5 added 215 characters in body edited Sep 23 '12 at 16:13 chris 12.7k44 gold badges4545 silver badges121121 bronze badges 4 building example on doc edited Sep 23 '12 at 16:06 chris 12.7k44 gold badges4545 silver badges121121 bronze badges 3 added 115 characters in body edited Sep 23 '12 at 15:56 chris 12.7k44 gold badges4545 silver badges121121 bronze badges 2 deleted 2 characters in body edited Sep 23 '12 at 15:39 chris 12.7k44 gold badges4545 silver badges121121 bronze badges 1 answered Sep 23 '12 at 15:34 chris 12.7k44 gold badges4545 silver badges121121 bronze badges