I observed that you included a timing before the plot, you missed a timing after the plot. Moreover, it appeared to me that TimeObject has quite an overhead, so I order to get "purer" timings, I replaced it by AbsoluteTime[]. This is what I used for timings:

points = 100;
f = {x, y} \[Function] x y;

opts = {PlotPoints -> points, MaxRecursion -> 0};
timing = Module[{i, start, results}, i = 0;
   results = Reap[
     Sow[{start = AbsoluteTime[], i++/points^2}];
     p = Plot3D[f[x,y], {x, -2, 2}, {y, -2, 2},
       Evaluate[opts],
       EvaluationMonitor :> Sow[{AbsoluteTime[], i++/points^2}]
       ];
     Sow[{AbsoluteTime[], i++/points^2}];
     ];
   {#1 - start, #2} & @@@ results[[2, 1]]];

Then I got this

So it's not for sure that there is no plateau for the normal-free case. I'd say that a major part of the plateaus in the end are due to the mesh lines because they get shorter with Mesh -> None. These are the timings without the mesh lines:

Moreover, there is some time needed to compute the polygon index lists for the final GraphicsComplex. This won't need any point evaluations (the points are already evaluated), so this will also contribute the the plateaus.

Notice also the small shift between the yellow and the first blue ramp. This could be the infamous 50 ms that might be due to compilation of the normal function into "WVM".

In the end, I still wonder what Mathematica is doing with the normal. In particular because you can get rid virtually all the time spent for "computing the vertex normals" by specifying your own NormalFunction like this:

νf = {x, y} \[Function] Evaluate[Cross @@ Transpose[D[{x, y, f[x, y]}, {{x, y}, 1}]]];
Plot3D[f[x, y], {x, -2, 2}, {y, -2, 2},NormalsFunction -> νf, Evaluate[opts]]

If the function f is sufficiently smooth, this will result in a perfect plot and the following timings

Addendum

If I needed many plots quickly, I would do the following:

points = 100;
mesh = {15, 15};
f = {x, y} \[Function] x y;
νf = {x, y} \[Function] Evaluate[-Cross @@ Transpose[D[{x, y, f[x, y]}, {{x, y}, 1}]]];

cfsurface = 
  With[{code = {Compile`GetElement[X, 1], Compile`GetElement[X, 2], 
      f[Compile`GetElement[X, 1], Compile`GetElement[X, 2]]}},
   Compile[{{X, _Real, 1}}, code, RuntimeAttributes -> {Listable}, Parallelization -> True]
   ];
cνf = 
  With[{code = νf[Compile`GetElement[X, 1], Compile`GetElement[X, 2]]},
   Compile[{{X, _Real, 1}}, code, RuntimeAttributes -> {Listable}, 
    Parallelization -> True]
   ];
getQuads = Compile[{{m, _Integer}, {n, _Integer}},
   Flatten[Table[
     {m (j - 1) + i, m (j - 1) + i + 1, m j + i + 1, m j + i},
     {j, 1, n - 1}, {i, 1, m - 1}], 1],
   CompilationTarget -> "C",
   RuntimeOptions -> "Speed"
   ];

points = 100;
pts2D = Tuples[{Subdivide[-2., 2., points - 1], Subdivide[-2., 2., points - 1]}];
lines = cfsurface@Outer[List, Subdivide[-2., 2., mesh[[1]] - 1], Subdivide[-2., 2., mesh[[2]] - 1]];
Graphics3D[{
  GraphicsComplex[
   cfsurface[pts2D], {EdgeForm[], Orange, Specularity[White, 30], 
    Polygon[getQuads[points, points]]}, 
   VertexNormals -> c\[Nu]f[pts2D]],
  Line@lines,
  Line@Transpose[lines]
  },
 Lighting -> "Neutral"
 ]

Even with compilation time included, this takes just 0.1 seconds (and about 0.018 seconds without compilation time).