Why not work with symbolic vectors? For instance, let:
x = y - z + ϵ s;
e = x / x.x;
e //TeXForm
$(y - z + s ϵ)/(y - z + s ϵ).(y - z + s ϵ)$
Then, compute the series by hand:
e0 = e /. ϵ -> 0;
e1 = D[e, ϵ] /. ϵ -> 0;
e2 = D[e, ϵ, ϵ]/2 /. ϵ -> 0;
e0 //TeXForm
e1 //TeXForm
e2 //TeXForm
$\frac{y-z}{(y-z).(y-z)}$
$\frac{s}{(y-z).(y-z)}-\frac{(y-z) (s.(y-z)+(y-z).s)}{((y-z).(y-z))^2}$
$\frac{1}{2} \left(\frac{2 (y-z) (s.(y-z)+(y-z).s)^2}{((y-z).(y-z))^3}-\frac{2 s (s.(y-z)+(y-z).s)}{((y-z).(y-z))^2}-\frac{2 s.s (y-z)}{((y-z).(y-z))^2}\right)$
Now, comparing e1 with your version:
r1 = (e1 /. {y->{y1,y2,y3,y4}, z->{z1,z2,z3,z4}, s->{s1,s2,s3,s4}})[[1]] //FullSimplify;
r1 //TeXForm
$\frac{\operatorname{s1} \left(-(\operatorname{y1}-\operatorname{z1})^2+(\operatorname{y2}-\operatorname{z2})^2+(\operatorname{y3}-\operatorname{z3})^2+(\operatorname{y4}-\operatorname{z4})^2\right)+2 (\operatorname{y1}-\operatorname{z1}) (\operatorname{s2} (\operatorname{z2}-\operatorname{y2})-\operatorname{s3} \operatorname{y3}+\operatorname{s3} \operatorname{z3}-\operatorname{s4} \operatorname{y4}+\operatorname{s4} \operatorname{z4})}{\left((\operatorname{y1}-\operatorname{z1})^2+(\operatorname{y2}-\operatorname{z2})^2+(\operatorname{y3}-\operatorname{z3})^2+(\operatorname{y4}-\operatorname{z4})^2\right)^2}$
r2 = SeriesCoefficient[
CapXp[{y1+ϵ s1,y2+ϵ s2,y3+ϵ s3,y4+ϵ s4},{y1,y2,y3,y4},{z1,z2,z3,z4}][[1]],
{ϵ,0,1}
] //FullSimplify;
r2 //TeXForm
$\frac{\operatorname{s1} \left(-(\operatorname{y1}-\operatorname{z1})^2+(\operatorname{y2}-\operatorname{z2})^2+(\operatorname{y3}-\operatorname{z3})^2+(\operatorname{y4}-\operatorname{z4})^2\right)+2 (\operatorname{y1}-\operatorname{z1}) (\operatorname{s2} (\operatorname{z2}-\operatorname{y2})-\operatorname{s3} \operatorname{y3}+\operatorname{s3} \operatorname{z3}-\operatorname{s4} \operatorname{y4}+\operatorname{s4} \operatorname{z4})}{\left((\operatorname{y1}-\operatorname{z1})^2+(\operatorname{y2}-\operatorname{z2})^2+(\operatorname{y3}-\operatorname{z3})^2+(\operatorname{y4}-\operatorname{z4})^2\right)^2}$
r1 === r2
True
Note that Series doesn't work correctly with Dot, but you can see this answer for a way to get Series to work properly with Dot, in which case you can just use Series instead. Here is the workaround from the answer:
protect = Unprotect[System`Private`InternalSeries];
System`Private`InternalSeries[a_Dot, {x_, x0_, n_Integer?NonNegative}] := Module[
{d = NestList[D[#, x]&, a, n], res},
res = Quiet @ Check[d /. x->x0, $Failed];
SeriesData[x, x0, TensorExpand @ res, 0, n+1, 1] /; res =!= $Failed
]
Protect @@ protect;
Then:
Series[e, {ϵ, 0, 1}] //Simplify //TeXForm
$\frac{y-z}{-y.z-z.y+y.y+z.z}+\frac{\epsilon ((y-z) (-s.y-y.s+s.z+z.s)+s (-y.z-z.y+y.y+z.z))}{(-y.z-z.y+y.y+z.z)^2}+O\left(\epsilon ^2\right)$
Update
For your revised expression you can use TensorProduct to represent $X_\mu X_\nu$. So:
e = With[{v = y - z + ϵ s}, TensorProduct[v,v]/v.v];
e/.ϵ->0
D[e,ϵ]/.ϵ->0//Simplify
D[e,ϵ,ϵ]/2/.ϵ->0//Simplify
I didn't use MathJax because it doesn't work well with \[TensorProduct].