I have large dataset and need to fit rather complicated function on it with different values of one of its parameters (this parameter must be fixed in every fit). I use the "LevenbergMarquardt"
algorithm of FindMinimum
as described here (with the only difference that my function is not a "black-box").
I notice that if I set
Method -> {"LevenbergMarquardt",
"Residual" -> Sqrt[2] residualVector[optimVariables],
"Jacobian" -> {"Symbolic", EvaluationMonitor :> ++steps}}
then FindMinimum
takes 5 minutes to create an optimized form of symbolic jacobian and after this every evaluation of the jacobian takes only 2 seconds.
I need to fit my function with many different values of the parameter and to spend 5 minutes for making the jacobian that is identical for all the fits (with exception for the value of only one parameter) is a waste of time. I tried to compute symbolic form of the jacobian by myself and got identical results with automatic symbolic jacobian. I used the code:
jacobianMatrix[_List?(VectorQ[#, NumberQ] &)] =
D[Sqrt[2] residualVector[optimVariables], {optimVariables}]
and
Method -> {"LevenbergMarquardt",
"Residual" -> Sqrt[2] residualVector[optimVariables],
"Jacobian" -> {jacobianMatrix[optimVariables], EvaluationMonitor :> ++steps}}
The problem is that each evaluation of this symbolic jacobian with numerical values of parameters takes 4 minutes! Obviously FindMinimum
optimizes its internal representation in some way and it gives huge speedup. But FindMinimum
creates the jacobian for every fit again. Is it possible to optimize the internal representation manually?
I should stress that I compute with WorkingPrecision
higher than MachinePrecision
. So probably compilation is not an option.
jacobianMatrix
once because high precision significantly decreases the number of steps needed to get the minimum: large data set results in large rounding-off errors in summation... $\endgroup$Experimental`NumericalFunction
, which is a structure produced byFindMinimum
from the function and its Jacobian that is optimized for fast numerical evaluation. I know almost nothing about these objects or how to create/use them, but I would like to find out. By the way, I am not sure if it is just a typo, but it is going to be a problem that you usedRuleDelayed
for the"Jacobian"
option in the second case. As it is currently, the 4 minute delay may be due to re-deriving the Jacobian at each point. $\endgroup$