# Expectation of a positive random variable gives negative results

Using Expectation to find $$E[\operatorname{score}^2]$$ evaluates to $$-\frac{1}{6}$$ below, why is this happening?

dist = ChiSquareDistribution[1];
shiftedDist = TransformedDistribution[x + t, x \[Distributed] dist];
pdf = PDF[shiftedDist, x];
score = D[Log@PDF[shiftedDist, x], t];
Expectation[score^2, x \[Distributed] shiftedDist] (* -1/6*)


I was using this to compute Fisher Information, and the wrong answers appear to happen for distributions with bounded domain, except for Uniform, where Expectation and NExpectation results coincide

ClearAll["Global*"];
fisher[dist_] := (
shiftedDist =
TransformedDistribution[x + \[Theta], x \[Distributed] dist];
pdf = PDF[shiftedDist, x];
score = \!$$\*SubscriptBox[\(\[PartialD]$$, $$\[Theta]$$]$$Log[ PDF[shiftedDist, x]]$$\);
Block[{\[Theta] =
0}, #[score^2, x \[Distributed] shiftedDist] & /@ {Expectation,
NExpectation}]
);

pairs = {
{"cauchy", CauchyDistribution[]},
{"normal", NormalDistribution[]},
{"student-t", StudentTDistribution[1]},
{"half-normal", HalfNormalDistribution[1]},
{"arcsin", ArcSinDistribution[{-1, 1}]},
{"Semicircle", WignerSemicircleDistribution[1]},
{"MarchenkoP", MarchenkoPasturDistribution[1]},
{"Exponential", ExponentialDistribution[1]},
{"Chi2", ChiSquareDistribution[1]},
{"uniform[0,1]", UniformDistribution[]},
{"uniform[-1,1]", UniformDistribution[{-1, 1}]},
{"bates", BatesDistribution[2]},
{"weihbul", WeibullDistribution[1/2, 2]}
};

TableForm[fisher[#[[2]]] & /@ pairs,
TableHeadings -> {First /@ pairs, {"fisher", "fisher n",
"variance"}}]

• -1. I see two syntax mistakes in your code: (i) x denotes the distribution ChiSquareDistribution[1] and the variable of PDF[shiftedDist, x]; (ii) score=D[Log@PDF[shiftedDist, x], t] is not any distribution at all. Jan 22 at 6:52
• Up to the definition, Fisher information depends on a parameter, but ChiSquareDistribution[1] has no parameter. The Maple's code with(Statistics): eval(FisherInformation(ChiSquare(n), 1, n), n = 1); results in $1/8\,{\pi}^{2}$. Jan 22 at 7:07
• @user64494 the stats part is discussed here, but regardless of how Fisher Information is defined, expected value of a positive variable shouldn't be negative Jan 22 at 7:25
• Sorry, I have nothing to discuss with you in such a non-constructive manner. I repeat there are two syntax errors in your code. That's all. Jan 22 at 8:13
• Good observation about whether the distribution is bounded. From en.wikipedia.org/wiki/Fisher_information: One of the required regularity conditions is "The support of f(X; θ) does not depend on θ."
– JimB
Jan 22 at 17:25

I wonder if the following explains the issue:

dist = ChiSquareDistribution[d];
shiftedDist = TransformedDistribution[x + t, x \[Distributed] dist];

pdf = PDF[shiftedDist, x][[1, 1, 1]]
(* (2^(-d/2) E^((t - x)/2) (-t + x)^(-1 + d/2))/Gamma[d/2] *)

score = (D[Log@PDF[shiftedDist, x], t] // PiecewiseExpand)[[1, 1, 1]]
(* (-2 + d + t - x)/(2 (t - x)) *)

fisherInfo = Integrate[score^2  pdf, {x, t, ∞}]
(* ConditionalExpression[((-2 + d) Gamma[-2 + d/2])/(8 Gamma[d/2]),
Re[t] > 0 && Im[t] == 0 && Re[d] > 4]
fisherInfo = FullSimplify[fisherInfo]
(* ConditionalExpression[1/(2 (-4 + d)), Re[t] > 0 && t == Re[t] && Re[d] > 4] *)


We see that we need $$d>4$$. So consider the following table:

Table[{d, 1/(2 (-4 + d))}, {d, 1, 6}] // TableForm


It appears that Mathematica is not considering the restriction that $$d>4$$.

• JimB (@ does not work): Can you kindly give us a reference to the definition of the Fisher information by fisherInfo = Integrate[score^2 pdf, {x, t, ∞}]? TIA. Jan 22 at 19:55
• Thanks for the PiecewiseExpand trick. Looks like Expectation takes some shortcuts, turning some other expectations into integrals in this way (WeibullDistribution[1/2, 2]) gives expected results Jan 22 at 20:43
• @user64494 en.wikipedia.org/wiki/Fisher_information. Under certain conditions (mentioned on that webpage) either the expectation of $\left(\frac{\partial}{\partial t}\log f(X;t)\right)^2$ or $-\frac{\partial^2}{\partial t^2}\log f(X;t)$ can be used (and result in the same value).
– JimB
Jan 23 at 2:27
• @JimB: Thank you . See my answer. Jan 23 at 5:54
• My comment about the possibility of closing this question because of statistical theory issues was wrong. Sorry about that. It does now appear that it's a Mathematica issue.
– JimB
Jan 23 at 5:58

Here is a calculation of the Fisher information for ChiSquareDistribution[d]. We start from

pdf = PDF[ChiSquareDistribution[d], t]


Piecewise[{{t^(-1 + d/2)/(2^(d/2)*E^(t/2)*Gamma[d/2]), t > 0}}, 0]

Now, according to the definition of the Fisher information,

FI = Integrate[D[Log[pdf], d]^2*pdf, {t, 0, Infinity},  Assumptions -> d >0]


1/4 PolyGamma[1, d/2]

and

FI /. d -> 1


Pi^2/8`

• What is incorrect in my answer? To downvote without any comment is not fair. Jan 23 at 6:54
• Note that table listed in question computes Fisher Information of TransformedDistribution with respect to shift parameter $t$ for various base distributions Jan 23 at 23:43