I observed that you included a timing _before_ the plot, you missed a timing _after_ the plot. Moreover, it appeared to me that `TimeObject` has quite an overhead, so I order to get "purer" timings, I replaced it by `AbsoluteTime[]`.  This is what I used for timings:

    points = 100;
    f = {x, y} \[Function] x y;

    opts = {PlotPoints -> points, MaxRecursion -> 0};
    timing = Module[{i, start, results}, i = 0;
       results = Reap[
         Sow[{start = AbsoluteTime[], i++/points^2}];
         p = Plot3D[f[x,y], {x, -2, 2}, {y, -2, 2},
           Evaluate[opts],
           EvaluationMonitor :> Sow[{AbsoluteTime[], i++/points^2}]
           ];
         Sow[{AbsoluteTime[], i++/points^2}];
         ];
       {#1 - start, #2} & @@@ results[[2, 1]]];

Then I got this

[![enter image description here][1]][1]

So it's not for sure that there is no plateau for the normal-free case. I'd say that a major part of the plateaus in the end are due to the mesh lines because they get shorter with `Mesh -> None`. These are the timings without the mesh lines:

[![enter image description here][2]][2]

Moreover, there is some time needed to compute the polygon index lists for the final `GraphicsComplex`. This won't need any point evaluations (the points are already evaluated), so this will also contribute the the plateaus. 

Notice also the small shift between the yellow and the first blue ramp. This could be the [infamous 50 ms](https://mathematica.stackexchange.com/questions/4700/shaving-the-last-50-ms-off-nminimize) that might be due to compilation of the normal function into `"WVM"`.

In the end, I still wonder what Mathematica is doing with the normal. In particular because you can get rid virtually all the time spent for "computing the vertex normals" by specifying your own `NormalFunction` like this:

    νf = {x, y} \[Function] Evaluate[Cross @@ Transpose[D[{x, y, f[x, y]}, {{x, y}, 1}]]];
    Plot3D[f[x, y], {x, -2, 2}, {y, -2, 2},NormalsFunction -> νf, Evaluate[opts]]

[![enter image description here][3]][3]

If the function `f` is sufficiently smooth, this will result in a perfect plot and the following timings

[![enter image description here][4]][4]


#Addendum

If I needed many plots (without mesh lines) _quickly_, I would do the following:

    f = {x, y} \[Function] x y;
    νf = {x, y} \[Function] Evaluate[-Cross @@ Transpose[D[{x, y, f[x, y]}, {{x, y}, 1}]]];
    cfsurface = 
      With[{code = {Compile`GetElement[X, 1], Compile`GetElement[X, 2], 
          f[Compile`GetElement[X, 1], Compile`GetElement[X, 2]]}},
       Compile[{{X, _Real, 1}}, code, RuntimeAttributes -> {Listable}, 
        Parallelization -> True]
       ];
    cνf = 
      With[{code = νf[Compile`GetElement[X, 1], 
          Compile`GetElement[X, 2]]},
       Compile[{{X, _Real, 1}}, code, RuntimeAttributes -> {Listable}, 
        Parallelization -> True]
       ];
    getQuads = Compile[{{m, _Integer}, {n, _Integer}},
       Flatten[Table[
         {m (j - 1) + i, m (j - 1) + i + 1, m j + i + 1, m j + i},
         {j, 1, n - 1}, {i, 1, m - 1}], 1],
       CompilationTarget -> "C",
       RuntimeOptions -> "Speed"
       ];

    points = 100;
    pts2D = Tuples[{Subdivide[-2., 2., points - 1], 
        Subdivide[-2., 2., points - 1]}];
    Graphics3D[
     GraphicsComplex[
      cfsurface[pts2D],
      {EdgeForm[], Orange, Specularity[White, 30], 
       Polygon[getQuads[points, points]]},
      VertexNormals -> cνf[pts2D]
      ],
     Lighting -> "Neutral"
     ]

Even with compilation time included, this takes just 0.1 seconds (and about 0.017 seconds without compilation time).

  [1]: https://i.sstatic.net/jHUrV.png
  [2]: https://i.sstatic.net/S67l7.png
  [3]: https://i.sstatic.net/ANctx.png
  [4]: https://i.sstatic.net/mq3JE.png