One more possibility would be to directly manipulate lists of coefficients of polynomials. To do that, we need to define a few required operations:
SetAttributes[{add, mult}, Orderless];
add[c1_?VectorQ, c2_?VectorQ] := Total[PadRight[{c1, c2}]];
mult[{0}, c2_?VectorQ] := {0};
mult[c1_?VectorQ, c2_?VectorQ] := ListConvolve[c1, c2, {1, -1}, 0];
diff[{_}] := {0};
diff[c_?VectorQ] := Rest[c] Range[Length[c] - 1];
int[c_?VectorQ] := Prepend[c/Range[Length[c]], 0];
and with those in hand,
NestList[add[mult[{0, 0, 1/2, 0, -1/2}, diff[#]], int[mult[{1/8, 0, -5/8}, #]]] &, {1}, 5]
{{1}, {0, 1/8, 0, -5/24}, {0, 0, 9/128, 0, -77/192, 0, 385/1152},
{0, 0, 0, 75/1024, 0, -4563/5120, 0, 17017/9216, 0, -85085/82944},
{0, 0, 0, 0, 3675/32768, 0, -96833/40960, 0, 144001/16384, 0, -7436429/663552,
0, 37182145/7962624},
{0, 0, 0, 0, 0, 59535/262144, 0, -67608983/9175040, 0, 250881631/5898240, 0,
-108313205/1179648, 0, 5391411025/63700992, 0, -5391411025/191102976}}
Use FromDigits[Reverse[coeff], x]
to reconstitute the polynomials.