**Bug introduced in 10.0 and persisting through 11.0** (reported as CASE:3790525)

----

Here's a simple series:

    Sum[t^k DiscreteDelta[k]/k!,{k,0,Infinity}]

Mathematica says that it doesn't converge. But replacing `Infinity` by any positive integer I get `1`. The same is for the case when I remove `k!` or `t^k` from the expression.

What's going on? Shouldn't `k!` _improve_ convergence rather than lead to divergence?

Note also that Mathematica can solve this easily:

    Sum[f[k] DiscreteDelta[k], {k, 0, Infinity}]
    (* f[0] *)