Imagine I have a function called fun1 that returns a vector of numbers fun1[a,b,c,d,...]:=. {a,b,c,d,...} are the parameters to be determined by fitting fun1[a,b,c,d,...] to test data. How can I use Levenberg-Marquardt algorithm to minimize $$\left\|\boldsymbol{fun1[a,b,c,d,...]}-\boldsymbol{testdata}\right\|_{2}\text{?}$$

Is something like this correct?

findMinimum[fun1[a,b,c,d],Transpose[{param,init}],MaxIterations -> 500, 
 PrecisionGoal -> 8, StepMonitor :> ++steps, 
 Method -> {"LevenbergMarquardt", "Residual" -> Norm[fun1[a,b,c,d]-testvec]}]
  • $\begingroup$ This really should have an explicit example. $\endgroup$ – Daniel Lichtblau Sep 14 '18 at 14:57

FindMinimum shifts to Levenberg-Marquardt automagically if it detects a sum of squares as objective. But it needs initial conditions:

 Total[(fun1[a, b, c, d] - testvec)^2], 
 {{a, a0}, {b, b0}, {c,c0}, {d, d0}},
 MaxIterations -> 500,
 PrecisionGoal -> 8,
 StepMonitor :> ++steps

with hopefully good guesses a0, b0, c0, and d0.

Better try NonlinearModelFit.

| improve this answer | |

NonlinearModelFit and FindFit are exactly for this purpose and have a lot of useful extra features. Supported methods include "ConjugateGradient", "Gradient", "LevenbergMarquardt", "Newton", "NMinimize", and "QuasiNewton". Please see the examples given in the documentation.

Your example does not have a variable (something like $x$) and it seems Mathematica always expects you to have one. In this case I needed an extra wrapping function to get the order of operations correct:

fun1[a_, b_, c_, d_] := 
  Table[Total[a { i, b i^2, c i^3, d i^4}], {i, 10}]
data = fun1[1, -5, 1, -0.001];
param = {a, b, c, d};
init = {1, 0, 0, 0};
steps = 0;
fun2[x_?NumberQ, args__] := fun1[args][[x]]
fit = NonlinearModelFit[data, fun2[i, a, b, c, d], 
  Transpose[{param, init}], i, MaxIterations -> 500, 
  PrecisionGoal -> 8, StepMonitor :> ++steps, 
  Method -> "LevenbergMarquardt"]
ListPlot[{data, Table[fit[x], {x, 10}]}, Joined -> {False, True}, 
 PlotLegends -> {"Data", "Fit"}]
ListPlot[fit["FitResiduals"], PlotLabel -> "Residuals"]

enter image description here

As this was not so elegant, maybe FindMinimum is actually a more elegant way of achieving the goal in absence of variables :/

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.