`Series` acts on variables outside its argument?

Question

I am trying to work with Taylor series. What I am trying to do is to express $$ f(h) - f'(0) = f(h) - f'(h - h) $$ such that only derivatives of $f$ with the argument $h$ appear, for example $$ f(h) - f'(0) = f(h) - f'(h) + hf''(h) + \mathcal O(h^2) $$

I tried to do is this way:

f[h] - Series[f'[s - h], {h, 0, 2}] /. s -> h

Strangely, this gives me $$ \bigl(f(0) - f'(h)\bigr) - \bigl(f'(0) - f''(h)\bigr)h +\left(\frac{f''(0)}2 - \frac{f^{(3)}(h)}2\right)h^2 + \mathcal O(h^2) $$ which is not what I wanted. How can this behaviour be explained?

---

For the record, the correct answer can be obtained by

f[t] - Series[f'[s - h], {h, 0, 2}] /. {t -> h, s -> h}

Answer 1

This is a bit tricky, but if you look in the docs under SeriesData, it says

When you apply certain mathematical operations to SeriesData objects, new SeriesData objects truncated to the appropriate order are produced.

That explains this interesting series of events:

Series[f[s + h], {h, 0, 2}]
g[h] - %
Series[g[h] - f[s + h], {h, 0, 2}]
Normal@% - Normal@%% // Expand

When I added g[h] to -Series[f[s + h], {h, 0, 2}] the system first converted g[h] to a SeriesData object with the same order and parameter.

You can do

f[h] - Normal@Series[f[s + h], {h, 0, 2}]
(* f[h] - f[s] - h Derivative[1][f][s] - 1/2 h^2 (f^′′)[s] *)

But that isn't really what you are looking for, since you still want the Big-O notation. Your method of working around this,

f[t] - Series[f'[s - h], {h, 0, 2}] /. {t -> h, s -> h}

(* (f(h)-f^′(h))+h f^′′(h)+1/2 h^2 (-f^(3)(h))+O(h^3) *)

works in the following way.

1. First, Series[f'[s - h], {h, 0, 2}] is evaluated, giving a SeriesData object.
2. Next, before f[t] can be added to -Series[f'[s - h], {h, 0, 2}], it is first converted to Series[f[t], {h, 0, 2}]
3. Now the two SeriesData objects are combined into one
4. The application of the replacement rule t->h occurs inside the SeriesData head, so there is not further application of Series