1
$\begingroup$

I am using FindFit function in order to fit my data and get two parameters: c and m.

The function that I am using has the following form:

function = (m/x*(x/c)^m)*Exp[-1*(x/c)^m];

The answer should be c = 64.68 and m = 2.47, but I am constantly getting the error message Overflow.

My data and core are here:

data = {{172.345, 0.000710716}, {136.899, 0.00238143}, {108.742, 
  0.00822192}, {86.3772, 0.0117385}, {68.6119, 0.0123856}, {54.5003, 
  0.0147863}, {43.2912, 0.0148628}, {34.3874, 0.0131214}, {27.3149, 
  0.00879578}, {21.697, 0.00634049}, {17.2345, 0.00310614}, {13.6899, 
  0.00231347}, {10.8742, 0.000858576}, {8.63772, 0.000286827}}; 

function = (m/x*(x/c)^m)*Exp[-1*(x/c)^m];

params = {c, m};
FindFit[data, function, params, x, MaxIterations -> 10000]

fitedfunc = function /. %

plot1 = ListPlot[data, PlotStyle -> Black, PlotMarkers -> "*"];
plot2 = Plot[fitedfunc, {x, 0, 200}, PlotStyle -> {Red, Thick}];
Show[plot1, plot2, Frame -> True]
$\endgroup$
  • $\begingroup$ I would start by specifying constraints on c and m because neither should be allowed to be negative as far as I can see. So something like FindFit[data, {function, c>0 && m > 0}, ...]. Maybe provide some starting values for the parameters as well, since non-linear fits tend to need decent starting guesses. $\endgroup$ – Sjoerd Smit Oct 2 at 20:05
  • $\begingroup$ Thx for your advice! I will try it. $\endgroup$ – Cro Simpson2.0 Oct 2 at 20:13
  • $\begingroup$ Taking the log of the expression and the data might help avoid overflow $\endgroup$ – mikado Oct 2 at 20:44
  • 2
    $\begingroup$ You have bigger problems than just overflow. Your curve is a probability distribution with an area under the curve of always 1 no matter what values of $m$ and $c$ you try. But connecting the data points has an area around 10 or 11. Also, it is very suspicious that the sum of the response values is 0.999099 (or probably would be exactly 1 if rounding wasn't involved). So it seems you've massaged the data in some fashion. So come clean. Are you needing to perform a regression where the shape just happens to be proportional to a probability distribution? Or fitting a real pdf? $\endgroup$ – JimB Oct 3 at 4:11
  • 2
    $\begingroup$ The ratios of neighboring x-values are all constant: Ratios[data[[All, 1]]] results in {0.794328, 0.794328, 0.794328, 0.794328, 0.794328, 0.794328, 0.794328, 0.794328, 0.794328, 0.794328, 0.794328, 0.794328, 0.794328}. That is consistent with the data consisting of relative histogram counts on a regular grid on a log scale from what is presented above. That likely means you really need to estimate a probability distribution from histogram counts (which have been lost by providing relative frequencies) on a log scale. $\endgroup$ – JimB Oct 3 at 4:36
3
$\begingroup$

A simple solution without rationalization and with standard accuracy

data = {{172.345, 0.000710716}, {136.899, 0.00238143}, {108.742, 
    0.00822192}, {86.3772, 0.0117385}, {68.6119, 0.0123856}, {54.5003,
     0.0147863}, {43.2912, 0.0148628}, {34.3874, 0.0131214}, {27.3149,
     0.00879578}, {21.697, 0.00634049}, {17.2345, 
    0.00310614}, {13.6899, 0.00231347}, {10.8742, 
    0.000858576}, {8.63772, 0.000286827}};

function = (m/x*(x/c)^m)*Exp[-(x/c)^m];

params = {{c, 64.}, {m, 2.5}};
f = FindFit[data, function, params, x]

fitedfunc = function /. f

plot1 = ListPlot[data, PlotStyle -> Black, PlotMarkers -> "*"];
plot2 = Plot[fitedfunc, {x, 0, 200}, PlotStyle -> {Red, Thick}];
Show[plot1, plot2, Frame -> True]
(*{c -> 71.0866, m -> 2.47334}*)

(*0.0000650406 E^(-0.0000262967 x^2.47334) x^1.47334*)

Note that this is slightly different from the solution @Anton Antonov Figure 1

$\endgroup$
  • $\begingroup$ Looks good! (+1) $\endgroup$ – Anton Antonov Oct 3 at 13:38
3
$\begingroup$

In order to avoid the overflow you can rationalize the numbers and increase the working precision.

In order to get the process going in the direction you want you can put constraints.

(Also, you might try to experiment using NonlinearModelFit.)

data2 = Map[Rationalize[#, 10^-12] &, data, {-1}];

pars = FindFit[data2, {function, 70 > c > 20, 4 > m > 1}, params, x, 
      WorkingPrecision -> 40, MaxIterations -> 1000]

(* {c -> 70.00000000000000000000000000000000000000,  
    m -> 2.488039920927348269685808190843090415001} *)

fitedfunc = function /. pars;
plot1 = ListPlot[data, PlotStyle -> Black, PlotMarkers -> "*"];
plot2 = Plot[fitedfunc, {x, 0, 200}, PlotStyle -> {Red, Thick}];
Show[plot1, plot2, Frame -> True]

enter image description here

$\endgroup$
  • $\begingroup$ Thank you very much for your answer Anton! $\endgroup$ – Cro Simpson2.0 Oct 3 at 12:30
3
$\begingroup$

This is an extended comment. You do not have a situation where a regression is an appropriate analysis (other than maybe to get starting values).

It appears (from the original data and your comment that you are sampling from a distribution) that you have a sample from which you created a histogram on a log scale of the sampled variable. Then you provided the histogram midpoints (back-transformed to the original scale) with the frequency counts turned into relative frequencies. That completely loses the sample size of the original sample. The sample size is the number of samples and NOT 14.

What should you have done? Your function is the probability density function of a Weibull distribution so use the raw data and

FindDistributionParameters[rawData, WeibullDistribution[m,c]]

If you only have the histogram frequencies (and by frequencies I mean integer counts), then one can still estimate the parameters. One would need to use the LogLikelihood and CDF[WeibullDistribution[m,c],x] functions. If you supply that data, I can give you the associated lines of Mathematica code. (And it would just be a few lines of code.)

While the other two answers are great if this were a regression problem, what you have is definitely not a regression problem. If you go that route I would not use those results in a journal article or justify the quality of a commercial product. That's how strongly I feel about this.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.