My apologies in so much as this is probably in large part a problem of me not being aware of the necessary mathematics but since this should be a common issue in machine learning and Mathematica now seems to have tools in this area, I'm hoping there is some simple way of doing this. Say I have some data set together with knowledge of how it changes when I vary some parameter

μ = 0;
data = Table[
RandomVariate[NormalDistribution[μ, σ], 10^3], {σ, 1, 100, 1}];

How can I find things like the Fisher Information Matrix? Of course I'm assuming that I do not know that this data set is obtained from the normal distribution. The idea is that this is just some black box which generates a data set and I can study how the data set changes when I change various input parameters.

  • 1
    $\begingroup$ This question may not be so natural. From wiki/Fisher_information, we have: "The Fisher Information is a way of measuring the amount of information that an observable random variable X carries about an unknown parameter θ upon which the probability of X depends." I'm not sure what the Fisher Information is supposed to mean in the nonparametric case (nonparametric because you mentioned you did want to assume that we knew we were dealing with normal distribution). $\endgroup$ – Jacob Akkerboom Jul 16 '14 at 14:35
  • 1
    $\begingroup$ I agree the scenario I have given here is a bit silly but I am mainly concerned with the practical issues regarding computing the FIM from a sample. If there are no built in methods I can use, then I have a question about an intermediate step: If the the data is drawn from a probability density function f(x;\sigma), then something I am confused about is how one estimates the derivative df/d\sigma. Because I have a finite sample, presumably I need to do some sort of smoothing of the data in order to avoid my estimate being dominated by noise. There must be some standard way of doing this. $\endgroup$ – user12876 Jul 16 '14 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.