You're not counting all of the parameters as you need to include the error variance. So with $n=5$ data points and 3 coefficients and one error variance the number of parameters is $k=4$ and $n-k-1=0$, hence the resulting infinity. (Variances are parameters, too!) See Hurvich & Tsai (1989) (*Biometrika* (1989), **76**, 2, pp. 297-307).