1
$\begingroup$

I am not even certain how to approach this problem.

  1. f[x] is a pdf: positive definite, normalized, and approaching 0 at +/- Infinity
  2. p[x] is f[x] convolved to a normal dist of mean = 0, variance = v.

I need to find the series of derivatives of the entropy wrt v in the limit v->0. The 0th term of the series is the entropy of f, and the 1th term is half the Fisher Information of f. The whole series is Expected values of sums of powers of derivatives of Log[f].

The coded definitions would just be:

nrml[x_, v_] := (1/Sqrt[2 Pi*v])*E^-((x^2)/(2 v)) p[x_, v_] := Integrate[nrml[x - xx, v]*f[xx], {xx, -Infinity, Infinity}] entropy[v_] := -Integrate[p[x, v]*Log[p[x, v]], {x, -Infinity, Infinity}]

But that gets tangled very quickly. Also, I want the final result to be in terms of Log[f] not f. So I tried to simplify matters by just working directly with g=Log[p] and leaving out the convolution completely; in other words I attempted to get everything in terms of g and in the end I can substitute g->Log[f] as v->0. And instead of making Mathematica do the full derivatives and substitute Normal[mean,0]->diracDelta, I just just this identity I derived on paper:

Derivative[n_, 1][g][x_, v_] := D[g[x, v], {x, n + 2}]/2 + D[D[g[x, v], x]^2/2, {x, n}]

I also tried leaving out the integration until the end:

entIntegrand[v_] := -g[x, v]*E^g[x, v] Expand[D[entIntegrand[v], v]]

But this is as far as I got because I don't know how to separate the terms of the integral, invoke some strategic integration by parts, or make use the endpoints of the integrand. This is straightforward on paper but extremely complicated, hence my trying to use Mathematica.

$\endgroup$
  • $\begingroup$ Can you define p[x] with some additional lines of code? $\endgroup$ – Jens Sep 25 '14 at 14:56
  • $\begingroup$ The coded definitions would just be: nrml[x_, v_] := (1/Sqrt[2 Pi*v])*E^-((x^2)/(2 v)) p[x_, v_] := Integrate[nrml[x - xx, v]*f[xx], {xx, -Infinity, Infinity}] entropy[v_] := -Integrate[ p[x, v]*Log[p[x, v]], {x, -Infinity, Infinity}] $\endgroup$ – Jerry Guern Sep 25 '14 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.