Calculate variance, standard deviation for conditional and marginal probability distributions

Variance and Standard Deviation for Marginal Probability Distributions

Generally, the variance for a joint distribution function of random variables \(X\) and \(Y\) is given by:

$$ Var\left(X,Y\right)=E\left(g\left({x}^2,\ {y}^2\right)\right)-\left(E\left[g\left(x,y\right)\right]\right)^2 $$

The standard deviation of joint random variables is the square root of the variance. Therefore, the standard deviation is given by:

$$ \sigma_{X,Y}=\sqrt{E\left(g\left({x}^2,\ {y}^2\right)\right)-\left(E\left[g\left(x,y\right)\right]\right)^2} $$

To determine the variance and standard deviation of each random variable that forms part of a multivariate distribution, we first determine their marginal distribution functions and compute the variance and the standard deviation, just like in the univariate case.

Variance of a Marginal Distribution (Discrete Case)

Suppose a joint distribution of the random variables \(X\) and \(Y\) is given by \(P_{XY}(X=x,\ Y=y)\) (typically abbreviated as \(P_{XY}(x,y)\)) for each pair \((x,y)\) of random variables, then for all discrete distributions, two vital requirements hold for each pair \((x,y)\):

$$ 0 \le P_{XY}\left(x,y\right)\le 1 $$

And,

$$ \sum_{all\ x}\sum_{all\ y}{P_{XY}\left(x,y\right)=1\ } $$

Then, the marginal probabilities \(P_X(X=x)\) and \(P_Y(Y=y)\) have variances \(Var(X)\) and \(Var(Y)\), respectively:

$$ Var\left(X\right)=E\left(X^2\right)-\left[E(X)\right]^2 $$

Where

$$ E\left(X^2\right)=\sum_{x}{x^2P(X=x)} $$

And

$$ E\left(X\right)=\sum_{x}{xP(X=x)} $$

And for the random variable \(Y\):

$$ Var\left(Y\right)=E\left(Y^2\right)-\left[E(Y)\right]^2 $$

Where

$$ E\left(Y^2\right)=\sum_{x}{y^2P(Y=y)} $$

And

$$ E\left(Y\right)=\sum_{x}{yP(Y=y)} $$

Standard Deviation of a Marginal Distribution (Discrete Case)

The standard deviation is the square root of variance. Denoted by \(\sigma_X\) and \(\sigma_Y\), respectively, the variance of \(X\) and \(Y\) is given by:

$$ \sigma_X=\sqrt{E\left(X^2\right)-\left[E(X)\right]^2} $$

And,

$$ \sigma_Y=\sqrt{E\left(Y^2\right)-\left[E(Y)\right]^2} $$

Example: Variance and Standard Deviation for Joint Random Variables (Discrete case)

Let \(X\) and \(Y\) have joint pmf:

$$ f\left(x,y\right)=\frac{x+y}{21},\ \ \ x=1,2,3\ \ \ \ \ y=1,2 $$

Calculate the variance and the standard deviation of \(X\).

Solution

We know that:

$$ Var\left(X\right)=E\left(X^2\right)-\left[E(X)\right]^2 $$

First, we find the marginal probability mass function of \(X\), which is given by:

$$ \begin{align*} P_X\left(x\right) & =\sum_{all\ x}{P\left(x,y\right)=}P\left(X=x\right),\ \ \ \ x\epsilon S_x \\ & =\frac{x+\left(1\right)}{21}+\frac{x+\left(2\right)}{21}=\frac{2x+3}{21} \end{align*} $$

Then,

$$ \begin{align*} E\left(X\right) & =\sum_{\left(all\ x\right)}{xP_X\left(x\right)} \\ & =\sum_{x=1}^{3}{xP_X\left(x\right)=\sum_{x=1}^{3}{x\frac{2x+3}{21}}} \\ & =\left(1\right)\frac{2\left(1\right)+3}{21}+\left(2\right)\frac{2\left(2\right)+3}{21}+\left(3\right)\frac{2\left(3\right)+3}{21}\\ &=1\left(\frac{5}{21}\right)+2\left(\frac{7}{21}\right)+3\left(\frac{9}{21}\right) \\ &=\frac{46}{21} \approx 2.1907619 \end{align*} $$

And

$$ \begin{align*} E\left(X^2\right) & =\sum_{all\ x}{x^2P_X\left(x\right)} \\ & =\left(1\right)^2\left(\frac{5}{21}\right)+\left(2\right)^2\left(\frac{7}{21}\right)+\left(3\right)^2\left(\frac{9}{21}\right)\\ & =\frac{38}{7} \end{align*} $$

Thus,

$$ \begin{align*} Var\left(X\right) & =E\left(X^2\right)-\left[E(X)\right]^2 \\ & =\frac{38}{7}-\left(\frac{46}{21}\right)^2=\frac{278}{441}\approx0.6304 \end{align*} $$

We know that the standard deviation of \(X\) is the standard deviation of its variance. Therefore,

$$ \begin{align*} \sigma_X & =\sqrt{Va\left(X\right)} \\ & =\sqrt{0.6304}=0.7840 \end{align*} $$

Variance of a Marginal Distribution (Continuous case)

We denote the pdf of a joint distribution of the random variables \(X\) and \(Y\) by \(f_{XY}(x,y)\). All continuous distributions must meet two main requirements for each ordered pair \((x,y)\) in the domain of \(f\). The properties include:

$$ f_{XY}\left(x,y\right)\geq 0 $$

And,

$$ \int_{x}\int_{y}{f_{XY}\left(x,y\right)=1} $$

Just like in the discrete case, we can calculate the variances by first calculating their marginal distributions, \(f_X(x)\) and \(f_Y(y)\). Recall that:

$$ f_X\left(x\right)=\int_{y}{f_{XY}\left(x,y\right)dy} $$

And

$$ f_Y\left(y\right)=\int_{x}{f_{XY}\left(x,y\right)dx} $$

Moreover, similar to the discrete case:

$$ Var\left(X\right)=E\left(X^2\right)-\left[E(X)\right]^2 $$

Where:

$$ E\left(X^2\right)=\int_{x}{x^2f_X\left(x\right)dx} $$

And

$$ E\left(X\right)=\int_{x}{xf_X\left(x\right)dx} $$

Similarly,

$$ Var\left(Y\right)=E\left(Y^2\right)-\left[E(Y)\right]^2 $$

Where

$$ E\left(Y^2\right)=\int_{y}{y^2f_Y\left(y\right)dy} $$

And

$$ E\left(Y\right)=\int_{y}{yf_Y\left(y\right)dy} $$

Standard Deviation of a Marginal Distribution (Continuous case)

We know that the standard deviation is the square root of the variance:

$$ \sigma_X=\sqrt{Var\left(X\right)} $$

And,

$$ \sigma_Y=\sqrt{Var\left(Y\right)} $$

Example: Variance and Standard Deviation for a Marginal Distribution (Continuous case)

Let \(X\) and \(Y\) have the joint pdf:

$$ f\left(x,y\right)=\left(\frac{4}{3}\right)\left(1-xy\right),\ \ \ \ \ 0\le x\le1,\ \ \ 0\le y\le1 $$

Find \(Var(Y)\) and \(\sigma_Y\).

Solution

First, let us find the marginal probability density for \(Y\).

We know that,

$$ \begin{align*} f_Y\left(y\right) & =\int_{x}{f_{XY}\left(x,y\right)dx} \\ & =\frac{4}{3}\int_{0}^{1}{\left(1-xy\right)\ \ dx} \\ & =\left(\frac{4}{3}\right)\left(1-\frac{y}{2}\right),\ \ \ 0\le y\le1\ \\ & =\frac{4}{3}\int_{0}^{1}{\left(1-xy\right)\ \ dx} \\ & =\left(\frac{4}{3}\right)\left(1-\frac{y}{2}\right),\ \ \ 0\le y\le1 \end{align*} $$

Then,

$$ \begin{align*} E\left(Y\right)& =\int_{y}{yf_Y\left(y\right)dy}=\int_{0}^{1}y\left(\left(\frac{4}{3}\right)\left(1-\frac{y}{2}\right)\right)dy \\ &=\left[\frac{4}{6}y^2-\frac{4y^3}{18}\right]_0^1 \\ &=\frac{4}{6}-\frac{4}{18} \\ & =\frac{4}{9} \end{align*} $$

Also,

$$ \begin{align*} E\left(Y^2\right) & =\int_{y}{y^2f_Y\left(y\right)dy}\\ &=\int_{0}^{1}{y^2\left(\frac{4}{3}\right)\left(1-\frac{y}{2}\right)\ dy}\\ &=\int_{0}^{1}{y^2\left(\frac{4}{3}\right)\left(1-\frac{y}{2}\right)\ dy}\\ &=\int_{0}^{1}{\frac{4y^2}{3}-\frac{4y^3}{6}dy}\\ &=\left|\frac{4y^3}{9}-\frac{4y^4}{24}\right|_0^1=\left(\frac{4}{9}-\frac{4}{24}\right) \\ & =\frac{5}{18} \end{align*} $$

Therefore,

$$ \begin{align*} & =\int_{0}^{1}{\frac{4y^2}{3}-\frac{4y^3}{6}dy=\left|\frac{4y^3}{9}-\frac{4y^4}{24}\right|_0^1=\left(\frac{4}{9}-\frac{4}{24}\right)=\frac{5}{18}} \\ Var \left(Y\right) & =E\left(Y^2\right)-\left[E\left(Y\right)\right]^2=\frac{5}{18}-\left(\frac{4}{9}\right)^2=\frac{13}{162} \approx 0.080246913 \\ \end{align*} $$

And now for finding the standard deviation:

$$ \begin{align*} \sigma_Y & =\sqrt{Var\left(Y\right)} \\ &=\sqrt{\frac{13}{162}}=0.2833=0.283278861 \\ \end{align*} $$

Variance and Standard Deviation for Conditional Distributions

Variance of a Conditional Distribution (Discrete Case)

The conditional variance of \(X\), given that \(Y=y\), is defined by:

$$ Var(X|Y=y)=E\left(X^2|Y=y\right)-\left[E(X|Y=y)\right]^2 $$

Where:

$$ E\left(X^2|Y=y\right)=\sum_{x}{x^2.g(x|Y=y)} $$

And

$$ E\left(X|Y=y\right)=\sum_{x}{x.g(x|Y=y)} $$

Note that this is analogous to the variance of a single random variable.

Similarly, the conditional variance of \(Y\) given \(X=x\) is defined by:

$$ Var(Y|X=x)=E\left(Y^2|X=x\right)-\left[E(Y|X=x)\right]^2 $$

Where:

$$ E\left(Y^2|X=x\right)=\sum_{y}{y^2.h(y|X=x)} $$

And

$$ E(Y|X=x)=\sum_{x}{y.h(y|X=x)} $$

Example: Conditional Variance (Discrete Case)

Let \(X\) and \(Y\) have a joint pmf:

$$ f\left(x,y\right)=\frac{x+y}{21},\ \ \ x=1,2,3\ \ \ \ \ y=1,2 $$

Find \(Var\left(X\middle| Y=1\right)\).

Solution

First, we have to find the marginal distribution \(f_Y(y)\) where we know that:

$$ \begin{align*} P_Y\left(y\right) & =\sum_{all\ y}{P\left(x,y\right)=P\left(Y=y\right),\ \ \ \ y\epsilon S_y} \\ {\Rightarrow P}_Y\left(y\right) & =\frac{\left(1\right)+y}{21}+\frac{\left(2\right)+y}{21}+\frac{\left(3\right)+y}{21}=\frac{6+3y}{21} \end{align*} $$

Secondly, the conditional distribution of \(X\) given that \(Y=y\), \(g\left(x\middle| y\right)\), is equal to:

$$ g\left(x\middle| y\right)=\frac{f\left(x,y\right)}{f_Y\left(y\right)}=\frac{\frac{x+y}{21}}{\frac{6+3y}{21}}=\frac{x+y}{3y+6} $$

We know that,

$$ Var\left(X\middle| Y=y\right)=\sigma_{X|Y=y}^2\ \ =E\left(X^2|Y=y\right)-\left[E(X|Y=y)\right]^2 $$

We need:

$$ Var(X|Y=1)=E\left(X^2|Y=1\right)-\left[E(X|Y=1)\right]^2 $$

We need to find \(E\left(X\middle| Y=1\right)\).

$$ \begin{align*} E\left(X\middle| Y=1\right) & =\sum_{x=1}^{3}{x.g\left(x\middle| y=1\right)} \\ & =\sum_{x=1}^{3}{x\frac{\left(x+\left(1\right)\right)}{3\left(1\right)+6}} \\ & =\left(1\right)\frac{\left(1+\left(1\right)\right)}{3\left(1\right)+6}+2\frac{\left(2+\left(1\right)\right)}{3\left(1\right)+6}+3\frac{\left(3+\left(1\right)\right)}{3\left(1\right)+6} \\ &=1\left(\frac{2}{9}\right)+2\left(\frac{1}{3}\right)+3\left(\frac{4}{9}\right) \\ & =\frac{20}{9} \end{align*} $$

Also, we need,

$$ \begin{align*} E\left(X^2|Y=y\right) & =\sum_{x=1}^{3}{x.g\left(x\middle| y=1\right)} \\ & =\sum_{x=1}^{3}{x^2\frac{\left(x+\left(1\right)\right)}{3\left(1\right)+6}} \\ & =\left(1^2\right)\frac{\left(1+\left(1\right)\right)}{3\left(1\right)+6}+2^2\frac{\left(2+\left(1\right)\right)}{3\left(1\right)+6}+3^2\frac{\left(3+\left(1\right)\right)}{3\left(1\right)+6} \\ & =1\left(\frac{2}{9}\right)+4\left(\frac{1}{3}\right)+9\left(\frac{4}{9}\right) \\ & =\frac{50}{9} \\ \Rightarrow Var\left(X\middle| Y=1\right) & =E\left(X^2|Y=1\right)-\left[E\left(X\middle| Y=1\right)\right]^2 \\ & =\frac{50}{9}-\left(\frac{20}{9}\right)^2=\frac{50}{81} \end{align*} $$

Standard Deviation of a Conditional Distribution (Discrete Case)

The conditional standard deviation of \(X\) given that \(Y=y\) is defined by:

$$ \sigma_{X|Y=y}=\sqrt{Var\left(X\middle| Y=y\right)} $$

While, the conditional standard deviation variance of \(Y\) given \(X=x\) is defined by:

$$ \sigma_{Y|X=x}=\sqrt{Var\left(Y\middle| X=x\right)} $$

Variance of a Conditional Distribution (Continuous Case)

The conditional variance of \(X\) given \(Y=y\) is:

$$ Var\left(X\middle| Y=y\right)=E\left(X^2\middle| Y=y\right)-\left[E\left(X\middle| Y=y\right)\right]^2 $$

Where:

$$ E\left(X^2\middle| Y=y\right)=\int_{x}{x^2g(x|Y=y)} $$

And

$$ E\left(X\middle| Y=y\right)=\int_{x}{x.g(x|Y=y)} $$

Using the same logic,

$$ Var\left(Y\middle| X=x\right)=E\left[Y^2\middle| X\right]-\left[E\left(Y\middle| X\right)\right]^2 $$

Where:

$$ E\left(Y^2\middle| X=x\right)=\int_{y}{y^2h(y|X=x)dy} $$

And

$$ E\left(Y\middle| X=x\right)=\int_{y}{y.h(y|X=x)dy} $$

Standard Deviation of a Conditional Distribution (Continuous Case)

$$ \sigma_{X|Y}=\sqrt{Var\left(Y\middle| X=x\right)} $$

And,

$$ \sigma_{Y|X}=\sqrt{Var\left(X\middle| Y=y\right)} $$

Example: Conditional Variance (Continuous case)

Let the joint pdf of random variables \(X\) and \(Y\) be given by:

$$ f\left(x,y\right)=\left(\frac{4}{3}\right)\left(1-xy\right),\ \ \ \ \ 0\le x\le1,\ \ \ 0\le y\le1 $$

Calculate \(Var\left(X\middle| Y=1\right)\).

Solution

First, let us find the marginal probability density for \(Y\).

We know that,

$$ \begin{align*} f_Y\left(y\right) & =\int_{x}{f_{XY}\left(x,y\right)dx} \\ & =\frac{4}{3}\int_{0}^{1}{\left(1-xy\right)\ \ dx} \\ & =\left(\frac{4}{3}\right)\left(1-\frac{y}{2}\right),\ \ \ 0\le y\le1\ \end{align*} $$

We also need we need the conditional distribution \(g(X|Y=y)\) given by:

$$ g\left(X\middle| Y=y\right)=\frac{f_{XY}(x,y)}{f_Y\left(y\right)}=\frac{\left(\frac{4}{3}\right)\left(1-xy\right)}{\left(\frac{4}{3}\right)\left(1-\frac{y}{2}\right)}=\frac{1-xy}{1-\frac{y}{2}} $$

We know that:

$$ Var\left(X\middle| Y=y\right)=E\left(X^2\middle| Y=y\right)-\left[E\left(X\middle| Y=y\right)\right]^2 $$

But we need:

$$ Var\left(X\middle| Y=1\right)=E\left(X^2\middle| Y=1\right)-\left[E\left(X\middle| Y=1\right)\right]^2 $$

$$ \begin{align*} E\left(X^2\middle| Y=1\right) & =\int_{x}{x^2.g\left(x\middle| Y=1\right)}dx \\ &=\int_{0}^{1}{x^2.\frac{1-(1)x}{1-\frac{1}{2}}=\int_{0}^{1}{2x^2(1-x)dx=\frac{1}{6}\ }} \end{align*} $$

Also, we need:

$$ \begin{align*} E\left(X\middle| Y=1\right) & =\int_{x}{x.g\left(x\middle| Y=1\right)}dx \\ &=\int_{0}^{1}{x.\frac{1-(1)x}{1-\frac{1}{2}}=\int_{0}^{1}{2x(1-x)dx=\frac{1}{3}\ }} \\ \Rightarrow Var\left(X\middle| Y=1\right) & =E\left(X^2\middle| Y=1\right)-\left[E\left(X\middle| Y=1\right)\right]^2 \\ &=\frac{1}{6}-\left(\frac{1}{3}\right)^2=\frac{1}{18} \end{align*} $$

Learning Outcome

Topic 3.e: Multivariate Random Variables – Calculate Variance, the standard deviation for conditional and marginal probability distributions.