# How to use Levenberg-Marquardt for parameter identification

Imagine I have a function called fun1 that returns a vector of numbers fun1[a,b,c,d,...]:=. {a,b,c,d,...} are the parameters to be determined by fitting fun1[a,b,c,d,...] to test data. How can I use Levenberg-Marquardt algorithm to minimize $$\left\|\boldsymbol{fun1[a,b,c,d,...]}-\boldsymbol{testdata}\right\|_{2}\text{?}$$

Is something like this correct?

findMinimum[fun1[a,b,c,d],Transpose[{param,init}],MaxIterations -> 500,
PrecisionGoal -> 8, StepMonitor :> ++steps,
Method -> {"LevenbergMarquardt", "Residual" -> Norm[fun1[a,b,c,d]-testvec]}]

• This really should have an explicit example. – Daniel Lichtblau Sep 14 '18 at 14:57

FindMinimum shifts to Levenberg-Marquardt automagically if it detects a sum of squares as objective. But it needs initial conditions:

FindMinimum[
Total[(fun1[a, b, c, d] - testvec)^2],
{{a, a0}, {b, b0}, {c,c0}, {d, d0}},
MaxIterations -> 500,
PrecisionGoal -> 8,
StepMonitor :> ++steps
]


with hopefully good guesses a0, b0, c0, and d0.

Better try NonlinearModelFit.

NonlinearModelFit and FindFit are exactly for this purpose and have a lot of useful extra features. Supported methods include "ConjugateGradient", "Gradient", "LevenbergMarquardt", "Newton", "NMinimize", and "QuasiNewton". Please see the examples given in the documentation.

Your example does not have a variable (something like $x$) and it seems Mathematica always expects you to have one. In this case I needed an extra wrapping function to get the order of operations correct:

fun1[a_, b_, c_, d_] :=
Table[Total[a { i, b i^2, c i^3, d i^4}], {i, 10}]
data = fun1[1, -5, 1, -0.001];
param = {a, b, c, d};
init = {1, 0, 0, 0};
steps = 0;
Dynamic[steps]
fun2[x_?NumberQ, args__] := fun1[args][[x]]
fit = NonlinearModelFit[data, fun2[i, a, b, c, d],
Transpose[{param, init}], i, MaxIterations -> 500,
PrecisionGoal -> 8, StepMonitor :> ++steps,
Method -> "LevenbergMarquardt"]
ListPlot[{data, Table[fit[x], {x, 10}]}, Joined -> {False, True},
PlotLegends -> {"Data", "Fit"}]
ListPlot[fit["FitResiduals"], PlotLabel -> "Residuals"]
fit["ParameterTable"] As this was not so elegant, maybe FindMinimum is actually a more elegant way of achieving the goal in absence of variables :/