I am curious if anybody has some experience in using the built-in Option "PenaltyFunction"
in any of the numerical optimization functions like NMinimize
.
According to the documentation the user should be able to provide NMinimize
with a custom penalty function to control the result of the minimization. The default option value is Automatic
. I tried to find some examples of how to use this Option
, but there is hardly any information available. The official documentation gives no example.
In the this Google.group post I found the following example, which is not working any more in Version 10.0.1.
NMinimize[{x + y + z, (1/20)*x + y + 5*z == 100, (x | y | z) ∈ Integers,
0 < x < 99, 0 < y < 99, 0 < z < 99}, {x, y, z},
Method -> {"SimulatedAnnealing", "SearchPoints" -> 250}, MaxIterations -> 500]
(* {55., {x -> 20, y -> 19, z -> 16}} *)
now with "PenaltyFunction"
NMinimize[{x + y + z, (1/20)*x + y + 5*z == 100, (x | y | z) ∈ Integers,
0 < x < 99, 0 < y < 99, 0 < z < 99}, {x, y, z},
Method -> {"SimulatedAnnealing", "PenaltyFunction" -> (100*(#1 - Floor[#1]) &),
"SearchPoints" -> 250}, MaxIterations -> 500]
This gives a warning of the solution not meeting the constraints and the result (which I don't understand):
(* {22., {x -> 20, y -> 1, z -> 1}}*)
I tried also to play with the setting myself:
From the documentation I took:
NMinimize[x + y, {x, y} ∈ Disk[]]
Show[ContourPlot[x + y, {x, y} ∈ Disk[]],
Graphics[{Red, PointSize[Large], Point[{x, y} /. Last[%]]}]]
Now with some PenaltyFunction
NMinimize[x + y, {x, y} ∈ Disk[], Method -> {"NelderMead",
"PenaltyFunction" -> (Min[#, 0] &)}]
Show[ContourPlot[x + y, {x, y} ∈ Disk[]],
Graphics[{Red, PointSize[Large], Point[{x, y} /. Last[%]]}]]
or
NMinimize[x + y, {x, y} ∈ Disk[], Method -> {"NelderMead",
"PenaltyFunction" -> ((# - Round[#])^3 &)}]
Show[ContourPlot[x + y, {x, y} ∈ Disk[]],
Graphics[{Red, PointSize[Large], Point[{x, y} /. Last[%]]}]]
I can't decide whether I am too stupid or just too lazy to figure out, what "PenaltyFunction"
actually does. Here my questions:
- What arguments does
"PenaltyFunction"
use/accept? - How is it possible to penalize individual fitting parameters?
- Do you have any example use cases that shed light on the whole issue?
EDIT
I found another Little Piece of Information here:
The author states that the Default Setting for "DifferentialEvolution"
is
"PenaltyFunction"-> Function[{d,i},d*10^(4*i)]
Function applied to penalize invalid Parameter values outside constraints (d
=distance
from allowed value, i
=number of iteration)
To me it is unclear if this applies to all constraints and how this is usefull.
NMinimize[x^2 - y^2, {x, y} \[Element] Disk[], Method -> {"SimulatedAnnealing", "PenaltyFunction" -> ((#1^4 + #2^4) &)}] Show[ContourPlot[x + y, {x, y} \[Element] Disk[]], Graphics[{Red, PointSize[Large], Point[{x, y} /. Last[%]]}]]
which would suggest it can take this as a penalty for points not satisfying the constraint? $\endgroup$ac = {}; NMinimize[{x^2 - y^2, x x + y y <= 1}, {x, y}, Method -> {"SimulatedAnnealing", "PenaltyFunction" -> ((AppendTo[ac, {##}]; 1) &)}] ; Print@ac
$\endgroup$"PenaltyFunction"
. For the unit disk, try"PenaltyFunction" -> (2(Norm[{##}] - 1) &)
. $\endgroup$"PenaltyFunction" -> (-(Norm[{##}] - 1) &)
$\endgroup$