@DanielLichtblau, you're right, it only happens when I use a = DiagonalMatrix[Table[10.^-10, {10000}]];, say. This is weird, but not terribly relevant in practice.

@DanielLichtblau (or others), the Do loop in logdet2 above seems to mess up the storing of the matrix in some way, causing CholeskyDecomposition to slow down enormously (for large matrices). I tried applying Developer`ToPackedArray but to no avail. Any ideas what is happening?

Good points @DanielLichtblau and Roman, I qualified the answer to say it is for symmetric p.d. matrices.

Hope the Wolfram team can add such a function soon!

Remarkable difference between windows and linux performance, anyone can shed more light on this?

Added some more output, is this what you needed?