We know that changing the option InterpolationOrder in ListLinePlot、ListPlot3D、ListContourPlot will change the shape of the curve:
(*A example from the help*)
data = {{0, 0}, {1, 2}, {3, 4}, {4, 2}, {6, 0}};
Table[ListLinePlot[data, InterpolationOrder -> n], {n, {0, 1, 3}}]

But nothing changes for NDSolve:
Table[Plot[Evaluate[First[x[t] /. NDSolve[{x'[t] == y[t], y'[t] == -x[t], x[0] == 1,
y[0] == 0}, {x, y}, {t, 0, 10}, InterpolationOrder -> n]]], {t, 0, 10}], {n, 1, 3}]

I checked the help and found the only example for InterpolationOrder in NDSolve is to set it to All. (The same example appears in two different places…) So I met the question written in the title, and, what if I want to change the InterpolationOrder of the InterpolatingFunction worked out by NDSolve?
(Sigh…) Seems that InterpolationOrder in NDSolve is really "useless", and there's no option that can change the interpolation order of InterpolatingFunction worked out by NDSolve…whatever! here I've already picked up another approach for changing the order: I just need to replace the InterpolatingPolynomial with Interpolation[…, InterpolationOrder -> …] in the link.