There are many fixes to this issue. I would recommend formulating problem from the start in terms of your constants. So you know exactly what constants mean.
F[x_, a_, b_] = y[x]
/. First@ DSolve[{x - y'[x] + y''[x] == 0, y[0] == a, y'[0] == b}, y[x], x];
Manipulate[Plot[F[x, a, b], {x, -10, 10}, PlotLabel -> F[x, a, b]],
{{a, -4, "initial function"}, -10, 10, Appearance -> "Labeled"},
{{b, .96, "initial 1st derivative"}, .5, 1.5, Appearance -> "Labeled"}]

For your more complicated case mentioned in the comments do this:
G[x_, a_, b_, c_] = {y[x], z[x]} /. First@DSolve[{y'[x] - 8*z'[x] == x^2,
z''[x] == x - y[x], y[0] == a, y'[0] == b, z[0] == c}, {y[x], z[x]}, x] // FullSimplify;
G[x, a, b, c] // Column // TraditionalForm

Manipulate[Plot[Evaluate@G[x, a, b, c], {x, -5, 5}, Filling -> 0,
PlotLabel -> Column[G[x, a, b, c]]],
{{a, 8, "initial y"}, -10, 10,Appearance -> "Labeled"},
{{b, 0, "initial y'"}, -10, 10, Appearance -> "Labeled"},
{{c, 0, "initial z"}, -10, 10, Appearance -> "Labeled"}]

Manipulate[ Plot[F[x] /. {C[1] -> a, C[2] -> b}, {x, -10, 10}], {a, 1, 6}, {b, -2, 5}]– b.gatessucks Aug 23 '12 at 19:40->is the operator form ofRule. When used in conjunction withReplaceAll(/.) (or an of the replace functions), the left-hand side (LHS) of theRuleis transformed into the right-hand side. Compare this toRuleDelayed(:>). – rcollyer Aug 23 '12 at 19:55