# Linearity of expectation not respected

I have found that Expectation gives different results if I use the fact that expectation is linear and evaluate it on every term independently.

I reduced my problem to the following code snippet. I use Mathematica 10.0.2:

f = x1 + x1 x2 z;
d = BernoulliDistribution[1/2 - z];
t = TransformedDistribution[
f, {x1 \[Distributed] d, x2 \[Distributed] d}];
e[x_] :=
Expand @ Simplify[Expectation[x, {x1 \[Distributed] d, x2 \[Distributed] t}]]
e[f]
Total[e /@ List @@ f]


Since expectation is linear, the last two expressions should evaluate to the same expression, but they don't. I get instead:

 1/2 - (3 z)/4 - (7 z^2)/8 + z^3/4 + (3 z^4)/2 - z^5

3/4 - (15 z)/8 + z^2/4 + (3 z^3)/2 - z^4


Is my understanding correct and this is a bug, or Mathematica is correct and I got it wrong?

• @Coolwater : are you sure? this lecture asserts that linearity of expectation holds even for dependent terms cse.iitd.ac.in/~mohanty/col106/Resources/… – M. Alaggan Feb 13 '16 at 12:00
• FullForm@f gives Plus[x1,Times[x1,x2,z]] and List @@ f gives {x1, x1 x2 z}. But the expectation of a product of independent random variables is the product of their expectations, not their sum. Expectation expects all listed distributions to be independent of each other. – Marco Breitig Feb 13 '16 at 15:27
• I don't know if this adds anything to the discussion but the two results become identical if z is set to some specific value between 0 and 1/2 (prior to defining f). – JimB Feb 13 '16 at 17:38

EXTENDED COMMENT. I cannot reproduce your problem. Which OS and Mathematica version are you using? Have you tried starting with a fresh kernel?

$Version (* "10.3.1 for Mac OS X x86 (64-bit) (December 9, 2015)" *) f = x1 + x1 x2 z; d = BernoulliDistribution[1/2 - z]; t = TransformedDistribution[f, {x1 \[Distributed] d, x2 \[Distributed] d}]; e[x_] := Expand@ Simplify[Expectation[x, {x1 \[Distributed] d, x2 \[Distributed] t}]] e[f] (* 1/2 - (3*z)/4 - (7*z^2)/8 + z^3/4 + (3*z^4)/2 - z^5 *) Total[e /@ List @@ f] (* 1/2 - (3*z)/4 - (7*z^2)/8 + z^3/4 + (3*z^4)/2 - z^5 *) % == %% (* True *)  • With Mathematica 10.0.2 on Win8 64 bit I get the same output as the OP. – Marco Breitig Feb 13 '16 at 15:12 • @MarcoBreitig - on my Mac it also works correctly with versions 9.0.1 and 10.3.0. Apparently the problem was introduced in version 10 and corrected in version 10.3.0 – Bob Hanlon Feb 13 '16 at 15:29 • What does List @@ f produce? See my other comment on the question. – Marco Breitig Feb 13 '16 at 15:32 • @MarcoBreitig - from the documentation for TransformedDistribution the form of the definition of t implies that the variables are independent. Were they dependent, the TransformedDistribution argument would use a bivariate or multivariate distribution for the variables. – Bob Hanlon Feb 13 '16 at 15:46 • @bon-hanlon: Yes, but I was arguing that E[X Y]=E[X] E[Y] for X, Y independent r.v., but Apply[List, X Y] gives {X,Y}. The expectations might be correctly calculated, but ´Total[e[X], e[Y]]´ is wrong if the initial expression was ´X Y´. – Marco Breitig Feb 13 '16 at 16:12 I have just had a quick squiz ... and express my concerns. First, I think such questions would be clearer if you first stated it in statistical/mathematical terms (not Mma code) ... and then provided your mma code. Then we can see if you are doing what you want to be doing. I am not quite sure what you want to be doing. Second, since t is defined in terms of TransformedDistribution which is a black box, how do you even know what you are doing, or linking to? To make this explicit, derive the pmf of the TransformedDistribution: d = BernoulliDistribution[1/2 - z]; t = PDF[TransformedDistribution[x1 + x1 x2 z, {x1 \[Distributed] d, x2 \[Distributed] d}], z]  returns: Piecewise[{{1/2, z == 0}, {-(3/4), z == 1}}, 0]  so t is just a function of parameter z, yet you are defining an expectation where  x2 \[Distributed] t  That looks like trouble to me. • For whatever it's worth, I wholeheartedly agree about providing some statistical background rather than just the Mathematica code. I assume you used z at the end of the PDF statement rather than some other symbol to show what trouble can arise. If one puts in say y instead of z, then the correct density is provided (assuming the constant z is between zero and 1/2). This makes t a density function rather than a distribution function. – JimB Feb 14 '16 at 19:34 I think the sequence of code is not constructed properly. First, we need to start with the correct expectation of f which we can do directly assuming that$x_1$and$x_2$are independent random variables distributed as$Bernoulli(1/2-z)\$:

$$E(f)=E(x_1+x_1 x_2 z)=E(x_1)+E(x_1)E(x_2)z=(1/2-z)+(1/2-z)(1/2-z)z=1/2-3z/4-z^2+z^3$$

Neither one of the two expressions in the question result in that value so something must be wrong with either my construction or the OP's.

If we do the following

f = x1 + x1 x2 z;
d = BernoulliDistribution[1/2 - z];
t = TransformedDistribution[f, {x1 \[Distributed] d, x2 \[Distributed] d}];


then t is the distribution of the random variable f and the expectation is

Expand[Expectation[f, f \[Distributed] t]]
(* 1/2 - (3 z)/4 - z^2 + z^3 *)


We could also call the random variable as y and get the same answer:

Expand[Expectation[y, y \[Distributed] t]]


But the function defined above in the question

e[x_] := Expand @ Simplify[Expectation[x, {x1 \[Distributed] d, x2 \[Distributed] t}]]


should not be expected to provide the correct expectation as for one thing x2 is not distributed with distribution t. But let's say that x1 has distribution d and x2 has distribution t with x1 and x2 independent (which by listing the two distributions in the format as you do, Mathematica expects those to be independent).

The expectation of x1 + x1 x2 z will be

Expand[(1/2 - z) + (1/2 - z) (1/2 - (3 z)/4 - z^2 + z^3) z]
(* 1/2 - (3 z)/4 - (7 z^2)/8 + z^3/4 + (3 z^4)/2 - z^5 *)


which is one of the outputs you found and that is not the desired expectation.