# Linear Combination of Normal Distributions

I'm just playing around with RandomVariate. Right now I can't really figure out why a linear combination with weights 0.5 of two identical normal distributions is not yielding the same distribution. 0.5*N($\mu$,$\sigma$)+0.5*N($\mu$,$\sigma$) should yield N($\mu$,$\sigma$).

This fails if I use a Monte Carlo approach.

    Map[(
\[Mu] = 1;
\[Sigma] = 0.4;
p1 = RandomVariate[NormalDistribution[\[Mu], \[Sigma]], 10^4];
p2 = RandomVariate[NormalDistribution[\[Mu], \[Sigma]], 10^4];
FindDistribution[0.5*p1 + 0.5*p2]
) &, Range[10]]


I'm pretty sure Mathematica is doing everything right. Still I can't really figure out why the StandardDeviation is far away from 0.4.

• Check TransformedDistribution[ 1/2 x + 1/2 y, {x \[Distributed] NormalDistribution[\[Mu], \[Sigma]], y \[Distributed] NormalDistribution[\[Mu], \[Sigma]]}] and see if you agree with its output. Aug 21, 2018 at 8:38
• @b.gatesucks Thanks for your comment. TransformedDistribution yields practically the same result as my approach.
– RMMA
Aug 21, 2018 at 8:56
• It would help if you were more specific with your terms. Are you interested in the linear combination of two normally distributed random variables, say, $Z=X/2+Y/2$? Or do are you interested in a mixture distribution where there are 3 random variables: $Z=\alpha X+(1-\alpha)Y$ where $X$ and $Y$ are normally distributed random variables (possibly independent of each other) and another Bernoulli random variable $\alpha$ that has $Pr(\alpha=1)=p$ and $Pr(\alpha=0)=1-p$ ?
– JimB
Aug 21, 2018 at 21:07

Formular for the variance of the linear combination of two randomvariables:

$\operatorname{Var}( a \,X + b \, Y) = a^2 \, \operatorname{Var}(X) + b^2 \, \operatorname{Var}(Y) + 2 \, a \, b \operatorname{Cov}(X,Y)$

For the standard deviations, you have to take the square root.

Your random variables are independent and have variances 0.4^2 and a = b = 0.5. So the resulting standard deviation is

Sqrt[a^2 0.4^2 + b^2 0.4^2 + 2 a b 0.]


0.282843

That's quite exactly what you obtain empirically from FindDistribution.

Lesson to learn: The distribution of the sum is not the sum of the distributions.

• Thanks, we are right.
– RMMA
Aug 21, 2018 at 9:19

I think the problem is the way you find the new sampled distribution. This is what you should do for testing you expression 0.5*N(μ,σ)+0.5*N(μ,σ) ~ N(μ,σ):

Map[(\[Mu] = 1;
\[Sigma] = 0.4;
p1 = RandomVariate[NormalDistribution[\[Mu], \[Sigma]], 10^4];
p2 = RandomVariate[NormalDistribution[\[Mu], \[Sigma]], 10^4];
{Mean@#, StandardDeviation@#} &@
MixtureDistribution[{0.5, 0.5}, {FindDistribution[p1],
FindDistribution[p2]}]
) &, Range[10]]


{{1.00193, 0.39969}, {1.00019, 0.402075}, {1.00143, 0.402793}, {0.997161, 0.399242}, {1.00011, 0.400222}, {1.00281, 0.401593}, {1.00001, 0.403046}, {1.00436, 0.400939}, {0.998176, 0.402365}, {1.00206, 0.396431}}

As you can see mean values and std devs are consistent with what you would expect.

• I do not really aggree, because what I want to get is a linear combination of normal distribution. This linear combination should be normally distributed itself. The mixture distribution is generally not a normal distribution.
– RMMA
Aug 21, 2018 at 8:55
• @RMMA from MMA documentation, it seems like MixtureDistribution[{w1,w2},{dist1,dist2}] does exactly what you are asking for: 0.5*N(μ,σ)+0.5*N(μ,σ). In your code you are not summing the distributions divided by 2, while you are summing the "0.5*data" sampled from the distributions and then you compute the new distribution from this result, which is not the same thing Aug 21, 2018 at 8:59
• It does and I appreciate your answer. Ultimately I would like to calculate the linear combination of arbritray normal distributions and I think MixtureDistribution is not the right way to go in this case. A quick look at the PDF shows that the MixtureDistribution of two normal distributions is generally no normal distribution. Contrary the linear combination of two normal distribution should be a normal distribution.
– RMMA
Aug 21, 2018 at 9:06
• Oh yes, I understand what you mean now, you want the general case (where mu and sigma might be different for the different distributions). In such a case I understand that the mixture doesn't help you much Aug 21, 2018 at 9:11