This function works indeed as expected, and we have
In[6]:= C3[1, {}, 1, {1, 1, 1, -4, -3}, 1, {}] // FullSimplify
Out[6]= 48 (3 C3[1, {}, 1, {-4}, 1, {}] + 5 (C3[1, {}, 1, {-3, -1}, 1, {}] + C3[1, {}, 1, {-2, -2}, 1, {}]))
Now I don't understand why an equivalent thing with an If does not work. If I use the function
C3p[a___, h2_, {n1___, m_, n_, n4___}, b___] := If[m > n,C3p[a, h2, {n1, n, m, n4}, b] + (m - n) C3p[a, h2, {n1, n + m, n4}, b] + If[m + n == 0, c (m^3 - m)/12 C3p[a, h2, {n1, n4}, b], 0], 0]
This one does not enter in the condition and we obtain
In[8]:= C3p[1, {}, 1, {1, 1, 1, -4, -3}, 1, {}]
Out[9]= 0