3
$\begingroup$

I have found that Expectation gives different results if I use the fact that expectation is linear and evaluate it on every term independently.

I reduced my problem to the following code snippet. I use Mathematica 10.0.2:

f = x1 + x1 x2 z;
d = BernoulliDistribution[1/2 - z];
t = TransformedDistribution[
   f, {x1 \[Distributed] d, x2 \[Distributed] d}];
e[x_] := 
  Expand @ Simplify[Expectation[x, {x1 \[Distributed] d, x2 \[Distributed] t}]]
e[f]
Total[e /@ List @@ f]

Since expectation is linear, the last two expressions should evaluate to the same expression, but they don't. I get instead:

 1/2 - (3 z)/4 - (7 z^2)/8 + z^3/4 + (3 z^4)/2 - z^5

 3/4 - (15 z)/8 + z^2/4 + (3 z^3)/2 - z^4

Is my understanding correct and this is a bug, or Mathematica is correct and I got it wrong?

$\endgroup$
  • $\begingroup$ @Coolwater : are you sure? this lecture asserts that linearity of expectation holds even for dependent terms cse.iitd.ac.in/~mohanty/col106/Resources/… $\endgroup$ – M. Alaggan Feb 13 '16 at 12:00
  • $\begingroup$ FullForm@f gives Plus[x1,Times[x1,x2,z]] and List @@ f gives {x1, x1 x2 z}. But the expectation of a product of independent random variables is the product of their expectations, not their sum. Expectation expects all listed distributions to be independent of each other. $\endgroup$ – Marco Breitig Feb 13 '16 at 15:27
  • $\begingroup$ I don't know if this adds anything to the discussion but the two results become identical if z is set to some specific value between 0 and 1/2 (prior to defining f). $\endgroup$ – JimB Feb 13 '16 at 17:38
3
$\begingroup$

EXTENDED COMMENT. I cannot reproduce your problem. Which OS and Mathematica version are you using? Have you tried starting with a fresh kernel?

$Version

(*  "10.3.1 for Mac OS X x86 (64-bit) (December 9, 2015)"  *)

f = x1 + x1 x2 z;

d = BernoulliDistribution[1/2 - z];

t = TransformedDistribution[f, {x1 \[Distributed] d, x2 \[Distributed] d}];

e[x_] := Expand@
  Simplify[Expectation[x, {x1 \[Distributed] d, x2 \[Distributed] t}]]

e[f]

(*  1/2 - (3*z)/4 - (7*z^2)/8 + z^3/4 + 
   (3*z^4)/2 - z^5  *)

Total[e /@ List @@ f]

(*  1/2 - (3*z)/4 - (7*z^2)/8 + z^3/4 + 
   (3*z^4)/2 - z^5  *)

% == %%

(*  True  *)
$\endgroup$
  • $\begingroup$ With Mathematica 10.0.2 on Win8 64 bit I get the same output as the OP. $\endgroup$ – Marco Breitig Feb 13 '16 at 15:12
  • $\begingroup$ @MarcoBreitig - on my Mac it also works correctly with versions 9.0.1 and 10.3.0. Apparently the problem was introduced in version 10 and corrected in version 10.3.0 $\endgroup$ – Bob Hanlon Feb 13 '16 at 15:29
  • $\begingroup$ What does List @@ f produce? See my other comment on the question. $\endgroup$ – Marco Breitig Feb 13 '16 at 15:32
  • $\begingroup$ @MarcoBreitig - from the documentation for TransformedDistribution the form of the definition of t implies that the variables are independent. Were they dependent, the TransformedDistribution argument would use a bivariate or multivariate distribution for the variables. $\endgroup$ – Bob Hanlon Feb 13 '16 at 15:46
  • $\begingroup$ @bon-hanlon: Yes, but I was arguing that E[X Y]=E[X] E[Y] for X, Y independent r.v., but Apply[List, X Y] gives {X,Y}. The expectations might be correctly calculated, but ´Total[e[X], e[Y]]´ is wrong if the initial expression was ´X Y´. $\endgroup$ – Marco Breitig Feb 13 '16 at 16:12
3
$\begingroup$

I have just had a quick squiz ... and express my concerns.

First, I think such questions would be clearer if you first stated it in statistical/mathematical terms (not Mma code) ... and then provided your mma code. Then we can see if you are doing what you want to be doing. I am not quite sure what you want to be doing.

Second, since t is defined in terms of TransformedDistribution which is a black box, how do you even know what you are doing, or linking to? To make this explicit, derive the pmf of the TransformedDistribution:

d = BernoulliDistribution[1/2 - z];
t = PDF[TransformedDistribution[x1 + x1 x2 z, {x1 \[Distributed] d, x2 \[Distributed] d}], z]

returns:

Piecewise[{{1/2, z == 0}, {-(3/4), z == 1}}, 0]

so t is just a function of parameter z, yet you are defining an expectation where

 x2 \[Distributed] t

That looks like trouble to me.

$\endgroup$
  • $\begingroup$ For whatever it's worth, I wholeheartedly agree about providing some statistical background rather than just the Mathematica code. I assume you used z at the end of the PDF statement rather than some other symbol to show what trouble can arise. If one puts in say y instead of z, then the correct density is provided (assuming the constant z is between zero and 1/2). This makes t a density function rather than a distribution function. $\endgroup$ – JimB Feb 14 '16 at 19:34
2
$\begingroup$

I think the sequence of code is not constructed properly. First, we need to start with the correct expectation of f which we can do directly assuming that $x_1$ and $x_2$ are independent random variables distributed as $Bernoulli(1/2-z)$:

$$E(f)=E(x_1+x_1 x_2 z)=E(x_1)+E(x_1)E(x_2)z=(1/2-z)+(1/2-z)(1/2-z)z=1/2-3z/4-z^2+z^3$$

Neither one of the two expressions in the question result in that value so something must be wrong with either my construction or the OP's.

If we do the following

f = x1 + x1 x2 z;
d = BernoulliDistribution[1/2 - z];
t = TransformedDistribution[f, {x1 \[Distributed] d, x2 \[Distributed] d}];

then t is the distribution of the random variable f and the expectation is

Expand[Expectation[f, f \[Distributed] t]]
(* 1/2 - (3 z)/4 - z^2 + z^3 *)

We could also call the random variable as y and get the same answer:

Expand[Expectation[y, y \[Distributed] t]]

But the function defined above in the question

e[x_] := Expand @ Simplify[Expectation[x, {x1 \[Distributed] d, x2 \[Distributed] t}]]

should not be expected to provide the correct expectation as for one thing x2 is not distributed with distribution t. But let's say that x1 has distribution d and x2 has distribution t with x1 and x2 independent (which by listing the two distributions in the format as you do, Mathematica expects those to be independent).

The expectation of x1 + x1 x2 z will be

Expand[(1/2 - z) + (1/2 - z) (1/2 - (3 z)/4 - z^2 + z^3) z]
(* 1/2 - (3 z)/4 - (7 z^2)/8 + z^3/4 + (3 z^4)/2 - z^5 *)

which is one of the outputs you found and that is not the desired expectation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.