Y[x_, n_] := Y[x, n] = -Simplify[
Integrate[(x - t)^(a - 1) (Y[x, n - 1] /. x -> t), {t, 0, x},
Assumptions -> 0 < a < 1, GenerateConditions -> False]/Gamma[a]]
If we set Y[x_, 0] = 1+r
then this yields the result $$y_n(x) = \frac{(-1)^n (1+r) x^{na}}{\Gamma(1+na)}.$$ Is this the answer you are looking for? In the case where you intended $y_0(x) = 1 + x$ instead of $1+r$, the result would be $$y_n(x) = \frac{(-1)^n (1+na+x)x^{na}}{\Gamma(2+na)}.$$ You can also do the whole thing as a single NestList
command:
NestList[-Simplify[
Integrate[(x - t)^(a - 1) (# /. x -> t), {t, 0, x},
Assumptions -> 0 < a < 1, GenerateConditions -> False]/
Gamma[a]] &, 1 + r, 10]
Once we know the form of $y_n(x)$, we can just create an explicit definition so that it is not necessary to do the computation:
Y[x_, n_, a_, r_] := (-1)^n (1 + r) x^(n a)/Gamma[1 + n a]
Then to get a plot for $a = 1/2$, $r = 2$, $n = 1, 2, \ldots, 10$, we just do something like
Plot[Evaluate[Table[Y[x, n, 1/2, 2], {n, 1, 10}]], {x, 0, 3}]
Or if you want to be fancy and make an interactive plot,
Manipulate[Plot[Y[x, n, 1/2, 2], {x, 0, 3}, PlotRange -> {-6, 6}], {n, 1, 10, 1}]
And for a two-column set of formulas for $a = 1/2$,
TableForm[Table[TraditionalForm /@ {Y[x, n, a, r], Y[x, n, 1/2, r]}, {n, 1, 10}]]
If you want TeX output, instead use
TeXForm[Table[{Y[x, n, a, r], Y[x, n, 1/2, r]}, {n, 1, 10}]]