Just a quick observation. Your integrand is not defined over some range, as can be seen by this plot
integrand = ((3.4641 (0.866025 +
r (-0.288675 + Sqrt[1 - 2/r + 0.01 r^2])))/(r^3 Sqrt[
1 - 2/r + 0.01 r^2] w)) // Rationalize
Plot[integrand /. w -> 1, {r, -5, 5}]
So to help Mathematica, tell it where the a and b are to avoid the problem area. Mathematica can do the indefinite integral OK
anti = Integrate[integrand, r]
Which gives one the clue the problem is with the limits given.
anti = Integrate[integrand, {r, a, b}, Assumptions -> {a > 2, b > a},
GenerateConditions -> False]
