# Why is multi-variable fitting in Mathematica so difficult?

Let me start by saying; yes this question has been asked in numerous ways countless times, however I still don't understand why fitting in Mathematica is so difficult.

I have the expression:

$P(\nu)=10Log_{10}(\frac{1}{\frac{1}{2\pi QL\nu_0} + \frac{1}{2i\pi \nu L C_{p}}+2i\pi \nu C_{p}})+P_{0}$

The real part of which I want to fit to my data and in so doing extract $Q,L,C_{p},P_{0}$ and $\nu_{0}$. My attempt at doing this in Mathematica is as follows:

FittedData = NonlinearModelFit[Data,10*Log10[Re[(1/(2*Pi*nu0*Q*L) + 1/(2*nu*Pi*L*I *Cp)+2*Pi*nu*Cp*I )^-1]] + P0, {{Q},{nu0},{Cp},{L},{P0}},nu]

Where I then try to see the results of the fit using:

FittedData["BestFitParameters"]

This fails. It only begins to work if I substitute some values in for the constants $Q,L,C_{p},P_{0}$ and $\nu_{0}$ for which I have a rough idea of from experience and some rough calculation. This feels self defeating as I want to extract these values in the first place.

My problem is that I really think there must be a way that Mathematica can spit all of these values out for me without having to "guess". For instance, if I create the same function in the analysis software OriginLab this will give me values close the ones which I would expect and with no "prompting" by me.

I have even tried playing with different methods for the fitting procedure, using Method->{"..."} but to no success.

What am I doing wrong that prevents Mathematica doing all of the work and determining these values for me?

PS If anyone would like the data, please let me know and I will add it.

The data:

29516700.00000,-20.39095
29517950.00000,-20.38086
29519200.00000,-20.38131
29520450.00000,-20.19700
29521700.00000,-19.99235
29522950.00000,-19.94068
29524200.00000,-19.89380
29525450.00000,-19.88163
29526700.00000,-19.76975
29527950.00000,-19.54841
29529200.00000,-19.57862
29530450.00000,-19.40761
29531700.00000,-19.37159
29532950.00000,-19.32529
29534200.00000,-19.04753
29535450.00000,-18.91629
29536700.00000,-18.91712
29537950.00000,-18.78629
29539200.00000,-18.63226
29540450.00000,-18.59667
29541700.00000,-18.58828
29542950.00000,-18.34601
29544200.00000,-18.29353
29545450.00000,-18.21738
29546700.00000,-17.99000
29547950.00000,-17.91704
29549200.00000,-17.76028
29550450.00000,-17.73089
29551700.00000,-17.59131
29552950.00000,-17.43442
29554200.00000,-17.36265
29555450.00000,-17.16377
29556700.00000,-17.05156
29557950.00000,-17.03910
29559200.00000,-16.76684
29560450.00000,-16.66412
29561700.00000,-16.52508
29562950.00000,-16.38831
29564200.00000,-16.26064
29565450.00000,-16.13360
29566700.00000,-15.97671
29567950.00000,-15.88092
29569200.00000,-15.74936
29570450.00000,-15.48040
29571700.00000,-15.33433
29572950.00000,-15.21547
29574200.00000,-15.06345
29575450.00000,-14.86060
29576700.00000,-14.75075
29577950.00000,-14.52547
29579200.00000,-14.30041
29580450.00000,-14.17481
29581700.00000,-13.99097
29582950.00000,-13.80429
29584200.00000,-13.64741
29585450.00000,-13.50066
29586700.00000,-13.32309
29587950.00000,-13.09386
29589200.00000,-12.82682
29590450.00000,-12.67758
29591700.00000,-12.44495
29592950.00000,-12.20982
29594200.00000,-11.93860
29595450.00000,-11.75053
29596700.00000,-11.49970
29597950.00000,-11.21890
29599200.00000,-10.98811
29600450.00000,-10.72840
29601700.00000,-10.46438
29602950.00000,-10.21057
29604200.00000,-9.90188
29605450.00000,-9.59494
29606700.00000,-9.33656
29607950.00000,-8.96102
29609200.00000,-8.62673
29610450.00000,-8.34331
29611700.00000,-7.94695
29612950.00000,-7.59639
29614200.00000,-7.23705
29615450.00000,-6.81505
29616700.00000,-6.41695
29617950.00000,-5.96252
29619200.00000,-5.47136
29620450.00000,-5.05636
29621700.00000,-4.57737
29622950.00000,-4.04987
29624200.00000,-3.50585
29625450.00000,-2.96981
29626700.00000,-2.38692
29627950.00000,-1.78203
29629200.00000,-1.17146
29630450.00000,-0.51088
29631700.00000,0.19580
29632950.00000,0.86940
29634200.00000,1.56243
29635450.00000,2.25264
29636700.00000,2.90473
29637950.00000,3.47042
29639200.00000,3.96090
29640450.00000,4.22117
29641700.00000,4.33517
29642950.00000,4.18756
29644200.00000,3.74489
29645450.00000,3.11859
29646700.00000,2.31640
29647950.00000,1.52684
29649200.00000,0.66335
29650450.00000,-0.13241
29651700.00000,-0.93485
29652950.00000,-1.60695
29654200.00000,-2.30148
29655450.00000,-2.91700
29656700.00000,-3.52910
29657950.00000,-4.05917
29659200.00000,-4.60647
29660450.00000,-5.11292
29661700.00000,-5.58141
29662950.00000,-6.05315
29664200.00000,-6.45304
29665450.00000,-6.83117
29666700.00000,-7.27239
29667950.00000,-7.60468
29669200.00000,-7.94312
29670450.00000,-8.30823
29671700.00000,-8.61603
29672950.00000,-8.96251
29674200.00000,-9.26712
29675450.00000,-9.56468
29676700.00000,-9.85901
29677950.00000,-10.12824
29679200.00000,-10.42618
29680450.00000,-10.71718
29681700.00000,-10.98261
29682950.00000,-11.18124
29684200.00000,-11.42554
29685450.00000,-11.65686
29686700.00000,-11.88427
29687950.00000,-12.11173
29689200.00000,-12.33115
29690450.00000,-12.54409
29691700.00000,-12.78411
29692950.00000,-12.91583
29694200.00000,-13.14108
29695450.00000,-13.33141
29696700.00000,-13.61789
29697950.00000,-13.76868
29699200.00000,-13.94983
29700450.00000,-14.12524
29701700.00000,-14.27209
29702950.00000,-14.46823
29704200.00000,-14.61564
29705450.00000,-14.81208
29706700.00000,-14.94302
29707950.00000,-15.10790
29709200.00000,-15.22393
29710450.00000,-15.46017
29711700.00000,-15.55410
29712950.00000,-15.55933
29714200.00000,-15.91787
29715450.00000,-15.96787
29716700.00000,-16.16727
29717950.00000,-16.30352
29719200.00000,-16.41175
29720450.00000,-16.60629
29721700.00000,-16.72901
29722950.00000,-16.86164
29724200.00000,-16.94622
29725450.00000,-17.17613
29726700.00000,-17.26062
29727950.00000,-17.37063
29729200.00000,-17.44479
29730450.00000,-17.59055
29731700.00000,-17.63568
29732950.00000,-17.86612
29734200.00000,-17.96255
29735450.00000,-18.03712
29736700.00000,-18.15099
29737950.00000,-18.18894
29739200.00000,-18.47704
29740450.00000,-18.53141
29741700.00000,-18.60920
29742950.00000,-18.61174
29744200.00000,-18.79474
29745450.00000,-19.01949
29746700.00000,-19.03202
29747950.00000,-19.13482
29749200.00000,-19.23107
29750450.00000,-19.28108
29751700.00000,-19.36718
29752950.00000,-19.57251
29754200.00000,-19.53079
29755450.00000,-19.58553
29756700.00000,-19.83076
29757950.00000,-19.89789
29759200.00000,-20.00088
29760450.00000,-20.10921
29761700.00000,-20.02411
29762950.00000,-20.23283
29764200.00000,-20.43967
29765450.00000,-20.38247
29766700.00000,-20.48476

• Data, please. Thank you. (But you might want to mention a package that you've found that has no trouble without having to give starting values for this particular function and data.)
– JimB
Sep 6 '16 at 0:11
• @JimBaldwin Hi Jim, thanks for replying, data added - sorry for the stupid format I couldn't fix that nicely. And there is no package, OriginLab is a completely separate software suite - sorry for any confusion!
– Q.P.
Sep 6 '16 at 0:18
• Thanks for the data. However, one issue is that I believe your model is overparametrized. Q and nu0 only appear as the product Q*nu0. As such you can only estimate their product.
– JimB
Sep 6 '16 at 0:51
• @JimBaldwin The problem is though I should be able to use this expression to extract them separately. In principle I know both of these values from when I make my measurement, however really they should be extracted from a fit. I would need to disentangle the product somehow.
– Q.P.
Sep 6 '16 at 0:56
• No, I don't think you should be able to do that. One can envision any of the parameters to be a product of two other parameters (or the product and/or sum of 6 other parameters). That doesn't mean that given the data and the function that one can figure out the two separate terms of the product.
– JimB
Sep 6 '16 at 1:03

I think that the example supplied doesn't justify the way the question is phrased. Fitting curves to data is not always easy for any known software (and the claim that a particular software package can deal with this data and the equation with no problem is lacking of any justification).

As pointed out by @george2079, there are only 3 "independent" parameters (and I'll suggest below that maybe even 3 is too many).

First, here's a slightly more compact form to provide the data:

y = {-20.39095, -20.38086, -20.38131, -20.197, -19.99235, -19.94068,
-19.8938, -19.88163, -19.76975, -19.54841, -19.57862, -19.40761,
-19.37159, -19.32529, -19.04753, -18.91629, -18.91712, -18.78629,
-18.63226, -18.59667, -18.58828, -18.34601, -18.29353, -18.21738,
-17.99, -17.91704, -17.76028, -17.73089, -17.59131, -17.43442,
-17.36265, -17.16377, -17.05156, -17.0391, -16.76684, -16.66412,
-16.52508, -16.38831, -16.26064, -16.1336, -15.97671, -15.88092,
-15.74936, -15.4804, -15.33433, -15.21547, -15.06345, -14.8606,
-14.75075, -14.52547, -14.30041, -14.17481, -13.99097, -13.80429,
-13.64741, -13.50066, -13.32309, -13.09386, -12.82682, -12.67758,
-12.44495, -12.20982, -11.9386, -11.75053, -11.4997, -11.2189,
-10.98811, -10.7284, -10.46438, -10.21057, -9.90188, -9.59494,
-9.33656, -8.96102, -8.62673, -8.34331, -7.94695, -7.59639, -7.23705,
-6.81505, -6.41695, -5.96252, -5.47136, -5.05636, -4.57737, -4.04987,
-3.50585, -2.96981, -2.38692, -1.78203, -1.17146, -0.51088, 0.1958,
0.8694, 1.56243, 2.25264, 2.90473, 3.47042, 3.9609, 4.22117,
4.33517, 4.18756, 3.74489, 3.11859, 2.3164, 1.52684,
0.66335, -0.13241, -0.93485, -1.60695, -2.30148, -2.917, -3.5291,
-4.05917, -4.60647, -5.11292, -5.58141, -6.05315, -6.45304, -6.83117,
-7.27239, -7.60468, -7.94312, -8.30823, -8.61603, -8.96251, -9.26712,
-9.56468, -9.85901, -10.12824, -10.42618, -10.71718, -10.98261,
-11.18124, -11.42554, -11.65686, -11.88427, -12.11173, -12.33115,
-12.54409, -12.78411, -12.91583, -13.14108, -13.33141, -13.61789,
-13.76868, -13.94983, -14.12524, -14.27209, -14.46823, -14.61564,
-14.81208, -14.94302, -15.1079, -15.22393, -15.46017, -15.5541,
-15.55933, -15.91787, -15.96787, -16.16727, -16.30352, -16.41175,
-16.60629, -16.72901, -16.86164, -16.94622, -17.17613, -17.26062,
-17.37063, -17.44479, -17.59055, -17.63568, -17.86612, -17.96255,
-18.03712, -18.15099, -18.18894, -18.47704, -18.53141, -18.6092,
-18.61174, -18.79474, -19.01949, -19.03202, -19.13482, -19.23107,
-19.28108, -19.36718, -19.57251, -19.53079, -19.58553, -19.83076,
-19.89789, -20.00088, -20.10921, -20.02411, -20.23283, -20.43967,
-20.38247, -20.48476};
x = Table[29516700 + 1250 i, {i, 0, 200}];
data = Transpose[{x, y}];


Now for the analysis:

{nuMin, nuMax} = MinMax[data[[All, 1]]];
nlm = NonlinearModelFit[data,
10 Log10[1/(a + b nuMax/nu + c nu/nuMax)], {a, b, c}, nu];
nlm["BestFitParameters"]
(* {a->-1.2512748397570334*^7,b->6.229896650560707*^6,
c->6.282964647767519*^6} *)

Show[ListPlot[data],
Plot[nlm[nu], {nu, nuMin, nuMax}, PlotStyle -> Red],
ImageSize -> Large]


But when the correlation matrix of the estimated parameters is examined, we see that the correlations are essentially either -1 or 1:

nlm["CorrelationMatrix"]
(* {{1.,-0.9999998531135882,-0.9999998531127312},
{-0.9999998531135882,0.9999999999999999,0.9999994124527141},
{-0.9999998531127312,0.9999994124527143,0.9999999999999998}} *)


This suggests that even a 3-parameter model is overparameterized.

Software packages should not be blamed for difficulties with overparameterized or poorly parameterized models. And good starting values and/or standardizing variables (both the response variable and predictor variables) are many times required. In addition, there is the issue of poorly parameterized models where large correlations (both negative and positive) among estimated coefficients can cause numerical problems.

Update: How to get back to the "original" model coefficients

Given that there are really only 3 parameters that can potentially be estimated from this model (aside from the error variance), consider the rewriting the model by dropping P0 and treating Q*nu0 as a single parameter. If a, b, and c are the estimates of the coefficients above, then we have

Solve[{a == 1/(2 Pi Qnu0 L),
b == 1/(2 I Pi L Cp),
c == 2 I Pi Cp}, {Qnu0, L, Cp}]
(* {{Qnu0 -> (b c)/(2 a π),
L -> 1/(b c),
Cp -> -((I c)/(2 π))}} *)

• "correlations are essentially either -1 or 1... suggests that even a 3-parameter model is overparameterized" Sorry for my poor basis for statistics, but can you elaborate a little on this part? Sep 6 '16 at 5:35
• @xzczd: Here is an example where the data is generated with just a single parameter (other than the error variance) but when we try a fit with two parameters (which is overparameterized for the underlying model generating the data), the correlation matrix is {{1,1},{1,1}}: x = Table[i/10, {i, 0, 20}]; y = 6 x + RandomVariate[NormalDistribution[0, 1], 21]; data = Transpose[{x, x, y}]; nlm = NonlinearModelFit[data, b z1 + c 2.3 z2, {b, c}, {z1, z2}]; nlm["BestFitParameters"] nlm["CorrelationMatrix"].We find those parameter estimators are perfectly correlated suggesting overparameterization.
– JimB
Sep 6 '16 at 6:14
• @JimBaldwin thanks for supplying a solution. In my specific case I can bundle some of the variables up becaue I have a reasonable idea of what they are. However, I still think the statement that Mathematica struggles to allow the user to fit data in an intuitive way stands. OriginLab which I have referred to before cab indeed fit this function with relative ease (granted I haven't provided evidence but suspend disbelief for the sake of discussion) my question really is about does anyone know how to make the fitting procedure less painful, e.g. use of methods and when to use which methods etc.
– Q.P.
Sep 6 '16 at 10:11
• @xzczd: Maybe a looser way to say things is that overparameterization can occur when (1) the model being fit has variable combinations such as Q*nu0 always together in the model in that form such that outside information is necessary to provide separate estimates, and (2) the actual data can't support too complex of a model. And I'm making up terms now: sort of a structural and data source of overparameterization, respectively.
– JimB
Sep 6 '16 at 13:23
• Maybe you have an older version of Mathematica. MinMax was introduced in version 10.1. You can replace that statement with nuMin = Min[data[[All,1]]]; nuMax=Max[data[[All,1]]].
– JimB
Sep 8 '16 at 14:15