4
$\begingroup$

Using Expectation to find $E[\operatorname{score}^2]$ evaluates to $-\frac{1}{6}$ below, why is this happening?

dist = ChiSquareDistribution[1];
shiftedDist = TransformedDistribution[x + t, x \[Distributed] dist];
pdf = PDF[shiftedDist, x];
score = D[Log@PDF[shiftedDist, x], t];
Expectation[score^2, x \[Distributed] shiftedDist] (* -1/6*)

I was using this to compute Fisher Information, and the wrong answers appear to happen for distributions with bounded domain, except for Uniform, where Expectation and NExpectation results coincide

enter image description here

ClearAll["Global`*"];
fisher[dist_] := (
   shiftedDist = 
    TransformedDistribution[x + \[Theta], x \[Distributed] dist];
   pdf = PDF[shiftedDist, x];
   score = \!\(
\*SubscriptBox[\(\[PartialD]\), \(\[Theta]\)]\(Log[
      PDF[shiftedDist, x]]\)\);
   Block[{\[Theta] = 
      0}, #[score^2, x \[Distributed] shiftedDist] & /@ {Expectation, 
      NExpectation}]
   );

pairs = {
   {"cauchy", CauchyDistribution[]},
   {"normal", NormalDistribution[]},
   {"student-t", StudentTDistribution[1]},
   {"half-normal", HalfNormalDistribution[1]},
   {"arcsin", ArcSinDistribution[{-1, 1}]},
   {"Semicircle", WignerSemicircleDistribution[1]},
   {"MarchenkoP", MarchenkoPasturDistribution[1]},
   {"Exponential", ExponentialDistribution[1]},
   {"Chi2", ChiSquareDistribution[1]},
   {"uniform[0,1]", UniformDistribution[]},
   {"uniform[-1,1]", UniformDistribution[{-1, 1}]},
   {"bates", BatesDistribution[2]},
   {"weihbul", WeibullDistribution[1/2, 2]}
   };

TableForm[fisher[#[[2]]] & /@ pairs, 
 TableHeadings -> {First /@ pairs, {"fisher", "fisher n", 
    "variance"}}]
$\endgroup$
13
  • $\begingroup$ -1. I see two syntax mistakes in your code: (i) x denotes the distribution ChiSquareDistribution[1] and the variable of PDF[shiftedDist, x]; (ii) score=D[Log@PDF[shiftedDist, x], t] is not any distribution at all. $\endgroup$
    – user64494
    Jan 22 at 6:52
  • $\begingroup$ Up to the definition, Fisher information depends on a parameter, but ChiSquareDistribution[1] has no parameter. The Maple's code with(Statistics): eval(FisherInformation(ChiSquare(n), 1, n), n = 1); results in $1/8\,{\pi}^{2}$. $\endgroup$
    – user64494
    Jan 22 at 7:07
  • $\begingroup$ @user64494 the stats part is discussed here, but regardless of how Fisher Information is defined, expected value of a positive variable shouldn't be negative $\endgroup$ Jan 22 at 7:25
  • $\begingroup$ Sorry, I have nothing to discuss with you in such a non-constructive manner. I repeat there are two syntax errors in your code. That's all. $\endgroup$
    – user64494
    Jan 22 at 8:13
  • $\begingroup$ Good observation about whether the distribution is bounded. From en.wikipedia.org/wiki/Fisher_information: One of the required regularity conditions is "The support of f(X; θ) does not depend on θ." $\endgroup$
    – JimB
    Jan 22 at 17:25

2 Answers 2

4
$\begingroup$

I wonder if the following explains the issue:

dist = ChiSquareDistribution[d];
shiftedDist = TransformedDistribution[x + t, x \[Distributed] dist];

pdf = PDF[shiftedDist, x][[1, 1, 1]]
(* (2^(-d/2) E^((t - x)/2) (-t + x)^(-1 + d/2))/Gamma[d/2] *)

score = (D[Log@PDF[shiftedDist, x], t] // PiecewiseExpand)[[1, 1, 1]]
(* (-2 + d + t - x)/(2 (t - x)) *)

fisherInfo = Integrate[score^2  pdf, {x, t, ∞}]
(* ConditionalExpression[((-2 + d) Gamma[-2 + d/2])/(8 Gamma[d/2]),
     Re[t] > 0 && Im[t] == 0 && Re[d] > 4]
fisherInfo = FullSimplify[fisherInfo]
(* ConditionalExpression[1/(2 (-4 + d)), Re[t] > 0 && t == Re[t] && Re[d] > 4] *)

We see that we need $d>4$. So consider the following table:

Table[{d, 1/(2 (-4 + d))}, {d, 1, 6}] // TableForm

Table of Fisher information statistics

It appears that Mathematica is not considering the restriction that $d>4$.

$\endgroup$
5
  • $\begingroup$ JimB (@ does not work): Can you kindly give us a reference to the definition of the Fisher information by fisherInfo = Integrate[score^2 pdf, {x, t, ∞}]? TIA. $\endgroup$
    – user64494
    Jan 22 at 19:55
  • $\begingroup$ Thanks for the PiecewiseExpand trick. Looks like Expectation takes some shortcuts, turning some other expectations into integrals in this way (WeibullDistribution[1/2, 2]) gives expected results $\endgroup$ Jan 22 at 20:43
  • 1
    $\begingroup$ @user64494 en.wikipedia.org/wiki/Fisher_information. Under certain conditions (mentioned on that webpage) either the expectation of $\left(\frac{\partial}{\partial t}\log f(X;t)\right)^2$ or $-\frac{\partial^2}{\partial t^2}\log f(X;t)$ can be used (and result in the same value). $\endgroup$
    – JimB
    Jan 23 at 2:27
  • $\begingroup$ @JimB: Thank you . See my answer. $\endgroup$
    – user64494
    Jan 23 at 5:54
  • $\begingroup$ My comment about the possibility of closing this question because of statistical theory issues was wrong. Sorry about that. It does now appear that it's a Mathematica issue. $\endgroup$
    – JimB
    Jan 23 at 5:58
-1
$\begingroup$

Here is a calculation of the Fisher information for ChiSquareDistribution[d]. We start from

pdf = PDF[ChiSquareDistribution[d], t]

Piecewise[{{t^(-1 + d/2)/(2^(d/2)*E^(t/2)*Gamma[d/2]), t > 0}}, 0]

Now, according to the definition of the Fisher information,

FI = Integrate[D[Log[pdf], d]^2*pdf, {t, 0, Infinity},  Assumptions -> d >0]

1/4 PolyGamma[1, d/2]

and

FI /. d -> 1

Pi^2/8

$\endgroup$
2
  • $\begingroup$ What is incorrect in my answer? To downvote without any comment is not fair. $\endgroup$
    – user64494
    Jan 23 at 6:54
  • $\begingroup$ Note that table listed in question computes Fisher Information of TransformedDistribution with respect to shift parameter $t$ for various base distributions $\endgroup$ Jan 23 at 23:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.