Why does Sow appear to miss many function calls in my code

Question

I encountered this problem while addressing an answer to this other question.

Here I do the following:

I ask NMaximize to do its job, and I use Reap and Sow with EvaluationMonitor to store the values that it tried in the list called reapSowPoints. At the same time, the objective function also stores the variables that were called in a separate list called functionCalls. These two lists should be the same, but they are actually very different. Why is that?

Here's the code:

myMaximization[vars_, objective_] := 
  Reap[
    Module[{customFunction},
      customFunction[v_ /; VectorQ[v, NumericQ]] := 
        Module[{val = objective @@ v}, 
          AppendTo[functionCalls, v]; 
          val];
        NMaximize[
          {customFunction[vars], Norm @ vars == 1}, 
          vars, 
          EvaluationMonitor :> Sow[vars]]]]

When I evaluate it I obtain (e.g.):

functionCalls = {};
{result, reapSowPoints} = myMaximization[{x, y}, #1 + #2 &];
Length /@ {functionCalls, reapSowPoints[[1]]}

{1603, 222}

The function was called 1603 times, but Sow appears to have caught only 222 of them.

Answer 1

The discrepancy is probably accounted for in post-processing the initial optimization routine. I couldn't check this without specifying an explicit method and post-processor.

myMaximization[vars_, objective_] := 
 Reap[Module[{customFunction}, 
   customFunction[v_ /; VectorQ[v, NumericQ]] := 
    Module[{val = objective @@ v},
     AppendTo[functionCalls, v];
     val];
   NMaximize[{customFunction[vars], Norm@vars == 1}, vars, 
    EvaluationMonitor :> Sow[vars, "NMaximize"], MaxIterations -> 100,
     Method -> {Automatic, 
      "PostProcess" -> {"FindMinimum", 
        EvaluationMonitor :> Sow[vars, "FindMinimum"]}}]],
  _,
  Rule]

functionCalls = {};
{result, reapSowPoints} = myMaximization[{x, y}, #1 + #2 &];
Length /@ {functionCalls, #[[1]], #[[2]], Join @@ #} &[{"NMaximize", 
   "FindMinimum"} /. reapSowPoints]
Length /@ DeleteDuplicates /@ 
  {functionCalls, #[[1]], #[[2]], Join @@ #} &[{"NMaximize",
    "FindMinimum"} /. reapSowPoints]
(*      Fn calls  NMaximize  FindMinimum  NMaximize+FindMinimum
  raw: {  1556,     222,        1126,            1348}
  DD:  {   577,     213,         379,            578}
*)

There's still a discrepancy in the raw data, but after the duplicates are removed, the discrepancy disappears. Many of the duplicates are the symbolic expression {x, y}, which alternates with a numeric pair in the post-processing calls. (I don't know why.) The symbolic {x, y} is not included in functionCalls because of the pattern check VectorQ[v, NumericQ] on customFunction[].