I have been trying to come op with a way to test if my dataset follows a Poisson distribution. Normally one would use the DistributionFitTest
function, but here is my problem. The way my data is collected, the mean of the distribution is not a constant, such that I have a dataset with entries "{$\lambda$,outcome}".
Ideally my data would look like,
dataperfect = {5# , RandomVariate[PoissonDistribution[5#]]} & /@ RandomReal[{0, 1}, 10^3];
Unfortunately I suspect there is a small deviation of my measured $\lambda$ and the true mean. So my data looks more like this,
datatest = {5#[[1]]+.5#[[2]], RandomVariate[PoissonDistribution[5#[[1]]+.5#[[2]]]]} & /@ RandomReal[{0, 1}, {10^3,2}];
My question, is there a way of determining how close my datatest
to a 'pure' Poisson distribution? Many thanks in advance.
EDIT
Since there is some discussion on whether the global or local comparison to a Poisson distribution is necessary some elaboration seems to be in order.
I have a certain model, a black box, which provides simultaneously two values $\lambda'_i$ and $\lambda''_i$ for the measured outcome's $\gamma'_i$ and $\gamma''_i$ of one event $i$. The assumption is made that the $\lambda_i$'s are Poisson distributed.
Further, a reasonable assumption is that the $\lambda_i$'s are a small deviation off the 'true' distribution parameter and the goal is to asses and minimise this deviation, by tweaking the model.
Using NonlinearModelFit
instead of GeneralizedLinearModelFit
in the way described here seems to do the trick, giving just a bit more freedom in defining a model for the expected deviation.
NonlinearModelFit
there is a strong caveat that must be mentioned. That function is much more flexible in the fixed effects part of the model but assumes that a least squares solution is appropriate and not what is optimal for a Poisson distribution. While that is partly mitigated by using an appropriateWeight
, that turns you farther away from being able to test for your original quest of "pure Poissonness". $\endgroup$