Determine the distribution of order st ...
Order Statistics Order Statistics are distributions obtained when we look at test scores... Read More
The n-th moment about the origin of a random variable is the expected value of its n-th power. Moments about the origin are \(E(X),E({ X }^{ 2 }),E({ X }^{ 3 }),E({ X }^{ 4 }),….\quad\)
For the most part, however, we are going to be looking at moments about the mean, also called central moments. The n-th central moment of a random variable \(X\) is the expected value of the n-th power of the deviation of \(X\) from its expected value.
Moments about the mean describe the shape of the probability function of a random variable.
Recall that the expected value of a random variable \(X\) is defined by
$$ E[X] = \sum_{x} {xp(x)} $$
where \(X\) is a discrete random variable with probability mass function \(p(x)\), and by
$$ E[X] = \int_{-\infty}^{\infty} xf(x)dx $$
when \(X\) is a continuous random variable with probability density function \(f(x)\). Since \(E[X]\) is a weighted average of the possible values of \(X\), it follows that if \(X\) must lie between a and b, then so must its expected value, i.e,.
If
$$ P(a\leq X \leq b) = 1 $$
Then
$$ a \leq E[X] \leq b $$
It follows that,
\(E [ag(X ) + bh(Y )] = aE [g(X )] + bE [h(Y )]\)
where a and b are constants.
What this equation tells us is that handling the expected value of a linear combination of functions is no more difficult than handling the expected values of the individual functions.
This handling also extends to situations where we have more than to variables. Expected values can easily be found from marginal distributions.
You have been given the following joint pmf. Verify that \(E[{ X }^{ 2 }+3Y]=E[{ X }^{ 2 }]+E[3Y]\)
$$ \begin{array}{c|c|c|c|c} {\begin{matrix} X \\ \huge{\diagdown} \\ Y \end{matrix}} & {0} & {1} &{2} \\ \hline {1} & {0.1} & {0.1} & {0} \\ \hline {2} & {0.1} & {0.1} & {0.2} \\ \hline {3} & {0.2} & {0.1} & {0.1} \end{array} $$
Solution
Reading values from the table, we have:
\((E[{ X }^{ 2 }+3Y]=0.1({ 0 }^{ 2 }+3\times 1)+0.1({ 0 }^{ 2 }+3\times 2)+0.2({ 0 }^{ 2 }+3\times 3)\)
\(+0.1({ 1 }^{ 2 }+3\times 1)+0.1({ 1 }^{ 2 }+3\times 2)+0.1({ 1 }^{ 2 }+3\times 3)\)
\(+0({ 2 }^{ 2 }+3\times 1)+0.2({ 2 }^{ 2 }+3\times 2)+0.1({ 2 }^{ 2 }+3\times 3)=8.1\)
Looking at the terms on the right side above,
\(E[{ X }^{ 2 }]={ 0 }^{ 2 }\times 0.4+{ 1 }^{ 2 }\times 0.3+{ 2 }^{ 2 }\times 0.3=1.5\)
\(E[3Y]=(3\times 1)0.2+(3\times 2)0.4+(3\times 3)0.4=6.6\)
Thus, \(E[{ X }^{ 2 }]+E[3Y]=8.1\), and the reasult has been verified.
Let \(X,Y\) be a pair of joint random variables with a joint probability function \(f(x,y)\) on the space \(S\). If there exists a function of these two namely \(g(X,Y)\) defined:
$$ E[g(X,Y)] = \sum_{(x,y) \in S} g(x,y)f(x,y) $$
Then this function is called the mathematical expectation (or expected value) of \(g(X,Y)\).
This mathematical expectation is known as the first moment of joint random variables, or mean.
The second moment is a derivative of the first moment and it is equal to:
$$ E[g(X,Y)]= E(g(X^2,Y^2)) – (E[g(X,Y)])^2 = Var(X,Y) $$
Let \(X\) and \(Y\) have the following pmf:
$$ f(x,y) = \frac{x^2 + 3y}{60} \qquad x = 1,2,3,4\quad y=1,2. $$
Find the expected value with \(g(X,Y) = XY\)
Solution
The possible values for this distribution are:
$$(1,1),(1,2),(2,1),(2,2),(3,1),(3,2),(4,1),(4,2)$$
Then we proceed to calculate,
$$\begin{align} E[XY] & = \sum_{(x,y) \in S} g(x,y) f(x,y)\\ & = \sum_{(x,y) \in S} (xy) \frac{x^2 + 3y}{60}\\ & = (1)\frac{1+3}{60} + (2)\frac{1+6}{60} + (2)\frac{4+3}{60} + (4)\frac{4+6}{60} + \\ & (3)\frac{9+3}{60} + (6)\frac{9+6}{60} + (4)\frac{16+3}{60} + (8)\frac{16+6}{60}\\ & = \frac{4}{60} + \frac{14}{60} + \frac{14}{60} + \frac{40}{60} + \frac{36}{60} + \frac{90}{60} + \frac{76}{60} +\frac{176}{60}\\ & = \frac{15}{2}=7.5 \end{align}$$
This process is quite similar to calculating the mean of any mass function, univariate or multivariate.
If \(X\) and \(Y\) are independent, then;
Let X and Y have the following pmf:
$$ f\left( x, y\right)=\frac{ x^2+3 y}{96}\ \ \ \ \ \ \ x=1,2,3,4\ \ \ \ y=1,2\ $$
Find the expected value of X and Y
Solution
To find the expected value of X, we need to find the marginal probability mass function of X which is given by;
$$\begin{align} f_{ x}\left( x\right) &=\sum_{ y}{ f\left( x, y\right)= P\left( X= x\right),\ x\epsilon S_{ x}} \\ f_{ X}\left( x\right) & =\frac{ x^2+3\left(1\right)}{96}+\frac{ x^2+3(2)}{96} \\ &=\frac{ x^2+3}{96}+\frac{ x^2+6}{96} \\ &=\frac{ x^2+ x^2+3+6 }{96} \\ \therefore\ f_X( x) & =\frac{2 x^2+9}{96} \end{align}$$
Therefore,
$$ \begin{align*} E\left( x\right) & =\sum_{ x=1}^{ n}{ {xf}_{ X}\left(x\right)\ \ \ \ \ \ \ } \\ & =\sum_{ x=1\ }^{4}{ {xf}_{ x}( x)} \\ & =\sum_{ x=1}^{4}{ x\frac{2 x^2+9}{96}\ } \\ & =\left(1\right)\frac{2\left(1\right)^2+9}{60}+\left(2\right)\frac{2\left(2\right)^2+9}{96}+\left(3\right)\frac{2\left(3\right)^2+9}{96}+\left(4\right)\frac{2\left(4\right)^2+9}{96} \\ & =\left(1\right)\frac{11}{96}+\left(2\right)\frac{17}{96}+\left(3\right)\frac{27}{96}+\left(4\right)\frac{41}{96}=\frac{145}{48}=3.02\ \end{align*} $$
Similarly, to find the expected value of Y, we need to find finding the marginal probability mass function of Y which is given by;
$$ \begin{align*} f_{ y}\left( y\right) & =\sum_{ x}{ f\left( x, y\right)= P\left( Y= y\right),\ \ \ y\epsilon S_{ y}} \\ & =\ \frac{1+3 y}{96}+\frac{4+3 y}{96}+\frac{9+3 y}{96}+\frac{16+3 y}{96} \\ & =\frac{12 y+30}{96} \\ \end{align*} $$
Therefore,
$$ \begin{align*} E\left( y\right) & =\sum_{ y=1}^{ n}{ {yf}_{ Y}\left( y\right)\ } \\ & =\sum_{ y=1}^{2}{ {yf}_{ Y}\left( y\right)} \\ & =\sum_{ y=1}^{2}{ y\frac{12 y+30}{96}} \\ & =\left(1\right)\frac{12\left(1\right)+30}{96}+\left(2\right)\frac{12\left(2\right)+30}{96}\ \\ &=\left(1\right)\frac{42}{96}+\left(2\right)\frac{54}{96}=\frac{25}{16}\ \end{align*} $$
We can also proceed to find the variance for the corresponding variables: For X, we know that:
$$ \text{Variance}= V\left( X\right)= E\left( X^2\right)-\left[ E\left( X\right)\right]^2 $$
Therefore, you need to find \( E( X^2)\) and \( E( X)\).
Continuing with the example above,
$$ \begin{align*} {Var}\left( X\right) & =\sum_{ x=1}^{4}{ x^2 f_{ x}\left( x\right)-\left[ E\left( X\right)\right]^2}\\ &=\sum_{ x=1}^{4}{ x^2\frac{2 x^2+9}{96}-\left(\frac{145}{48}\right)^2} \\ & =\left(1\right)^2\frac{11}{96}+\left(2\right)^2\frac{17}{96}+\left(3\right)^2\frac{27}{96}+\left(4\right)^2\frac{41}{96}-\left(\frac{145}{96}\right)^2=\frac{163}{16}-\left(\frac{145}{48}\right)^2=1.062\ \\ \text{ Similarly, for Y:}\\ {Var}\left( Y\right) & =\sum_{ y=1}^{2}{ y^2{ f}_{ y}\left( y\right)-\left[ E\left( Y\right)\right]^2} \\ & =\sum_{ y=1}^{2}{ y^2\frac{12 y+30}{96}-\left(\frac{25}{16}\right)^2} \\ & =\left(1\right)^2\frac{42}{96}+\left(2\right)^2\frac{54}{96}-\frac{625}{256}=\frac{43}{16}-\frac{625}{256}=\frac{63}{256}\ \end{align*} $$
Let \(X\) and \(Y\) have the following pmf:
$$ f(x,y) = \begin{cases} 3(x+y), & 0 < x \le y \le 1\\ 0, &\text{otherwise}\\ \end{cases} $$
Find the expected value of \(X\) and \(Y\)
Solution
To find the mean/expectation of X, we need to find the marginal distribution of X. We know that:
$$f_x\left(x\right)=\int_{-\infty}^{\infty}{f\left(x,y\right)dy, \ \ x\epsilon S_x}$$
Then
$$\begin{align} f_X\left(x\right) &=\int_{x}^{1}{\left(3x+3y\right)\ dy}=\left|3xy+\frac{3y^2}{2}\right|_x^1\\ &=\left(3x+\frac{3}{2}\right)-\left(3x^2+\frac{3x^2}{2}\right) =3x-\frac{9x^2}{2}+\frac{3}{2}\end{align}$$
Hence we know that:
$$E\left[X\right]=\int_{-\infty}^{\infty}{xf_X\left(x\right)\ dx}$$
Then,
$$\begin{align} E\left(X\right)&=\int_{0}^{1}{x\left(3x-\frac{9x^2}{2}+\frac{3}{2}\right)\ dx}\\ &=\left|\frac{3x^3}{3}-\frac{9x^4}{8}+\frac{3x^2}{4}\right|_0^1\\ &=1-\frac{9}{8}+\frac{3}{4}=\frac{5}{8}\\ &=1-\frac{9}{8}+\frac{3}{4}=\frac{5}{8} \end{align}$$
$$\therefore E\left(x\right)=\frac{5}{8} $$
For variance of X, we know that,
$$Var(X)=E\left(X^2\right)-\left[E\left(X\right)\right]^2$$
Now,
$$E\left(X^2\right)=\int_{0}^{1}{x^2\left(3x-\frac{9x^2}{2}+\frac{3}{2}\right)\ dx} =\frac{7}{10}$$
Thus,
$$Var(X) = E\left(X^2\right)-\left[E\left(X\right)\right]^2=\frac{7}{10}-\left(\frac{5}{8}\right)^2=\frac{99}{320}$$
Let \(X\) and \(Y\) be random variables with a joint probability function \(f(x,y)\) and marginal functions \(f_x(x)\) and \(f_y(y)\)
The conditional pmf of \(Y\) given that \(X=x\) is defined by:
$$h(y|x)=\cfrac { f(x,y) }{ { f }_{ X }(x) }\quad\quad\quad\quad\quad\text{provided that}\quad f_X(x)>0$$
The conditional mean of \(Y\), given that \(X=x\) is defined:
$$ \mu_{Y|x} = E[Y|x] = \sum_{y} y h(y|x), $$
and the conditional variance of \(Y\), given that \(X=x\) is defined:
$$ \sigma_{Y|x} = E\left\{(Y – E[Y|x])^2|x \right\} = \sum_{x}(y-E[Y|x])^2 h(y|x) $$
This is simplified to:
$$ \sigma^2_{Y|x} = E[Y^2|x] – (E[Y|x])^2 $$
Note also that:
$$h\left(y\middle| x\right)=\frac{f\left(x,y\right)}{f_X\left(x\right)}\ \ provided\ that\ f_X\left(x\right) >0$$
So that:
$$\mu_{x|y}=E[X|Y]=yxh(x|y)$$
And
$$\sigma_{x|y}^2=E\left[Y^2\middle| X\right]-E[X|Y]^2$$
The joint probability mass function of variables X and Y is given by:
$$f(x,y) = \frac{x^2 +3y}{60},\ x=1,2,3,4;\ y=1,2$$
Calculate :
a). E(X|Y=1)
b). E(Y|X=3)
c). V(X|Y=1)
Solution
From the joint function, we can get the following marginal pmfs:
$$f_X\left(x\right)=\frac{2x^2+9}{60}\ \ \text{and} \ f_Y\left(y\right)=\frac{12y+30}{60}$$
We can also find conditional probability mass function:
$$g\left(x\middle| y\right)=\frac{x^2+3y}{12y+30}\ \text{and}\ h\left(y\middle| x\right)=\frac{x^2+3y}{2x^2+9}$$
So,
a). Finding \(E(X|Y=1):
$$\begin{align} E\left(X\middle| Y=1\right)&=E(g(x|y=1))\\ &=\sum_{x=1}^{4}{xg\left(x\middle| y=1\right)}\\&=\sum_{x=1}^{4}{x\frac{x^2+3\left(1\right)}{12\left(1\right)+30}} \\ &=\frac{65}{21}=3.10\end{align}$$
b). Finding \(E(Y|X=3)\):
$$\begin{align} E\left(Y\middle| X=3\right)&=E(h(y|x=3))\\ &=\sum_{y=1}^{2}{yh(y|x=3)}\\ &=\sum_{y=1}^{2}{y\ \frac{3^2+3y}{2\left(3\right)^2+9}}\\&=\left(1\right)\frac{3^2+3\left(1\right)}{2\left(3\right)^2+9}+\left(2\right)\frac{3^2+3\left(2\right)}{2\left(3\right)^2+9}\\&=\frac{12}{27}+\left(2\right)\frac{15}{27}=\frac{14}{9}=1.56\end{align}$$
c). Finding \(Var(X|Y=1)\)
Using the output from (a), we have:
$$\begin{align}V\left(X\middle| Y=1\right)&=E\left[X-E\left(X\middle| Y=1\right)\right)^2|Y=1]\\ & =\sum_{x=1}^{4}{\left(x-E\left(X\middle| Y=1\right)\right)^2g(x|y=1)}\\ &=\sum_{x=1}^{4}{\left(x-\frac{65}{21}\right)^2\frac{x^2+3\left(1\right)}{12\left(1\right)+30}}\\ &=0.99 \end{align}$$
When \(X\) and \(Y\) are continuous random variables, the conditional pdf, mean, and variance are given as follows:
Conditional pdf:
$$ g(x|y)=\cfrac { f(x,y) }{ { f }_{ Y}(y) } \quad\quad\quad\quad\quad\text{provided that}\quad f_Y(y)>0$$
Conversely,
$$h\left(y\middle| x\right)=\frac{f\left(x,y\right)}{f_X\left(x\right)}\ \text{provided\ that}\ f_X\left(x\right) > 0$$
Conditional mean:
$$E(Y|X)=\int _{ -\infty }^{ \infty }{ yh(y|x)\partial y } $$
Also
$$E\left(Y\middle| X\right)=\int_{-\infty}^{\infty}{yh\left(y\middle| x\right)\partial y}$$
Conditional variance:
$$\text{Var}(Y|X)=E\{ [Y-E(Y|x)]^{ 2 }|x\} $$
$$=\int _{ -\infty }^{ \infty }{ [y-E(Y|x)]^{ 2 }h(y|x)\partial y } $$
$$=E[{ Y }^{ 2 }|x]-{ [E(Y|x] }^{ 2 }$$
Using the same logic,
$$Var\left(Y\middle| X\right)=E\left[Y^2\middle| X\right]-\left[E\left(Y\middle| X\right)\right]^2$$
Let
$$ f(x,y) = \frac{4}{3}(1-xy) \qquad 0\leq x\leq1,\quad 0\leq y\leq1 $$
Find:
a). g(x|y)
b). E(x|y=1)
Solution
a): Conditional function \(g(x|y)\)
We know that:
$$\begin{align} g\left(x\middle| y\right)&=\frac{f\left(x,y\right)}{f_Y\left(y\right)}\\ &=\frac{\frac{4}{3}\left(1-xy\right)}{\int_{0}^{1}{\frac{4}{3}\left(1-xy\right)dx}}\\&=\frac{\frac{4}{3}\left(1-xy\right)}{\frac{4}{3}{\left[x-\frac{x^2y}{2}\right]\left(x-\frac{x^2y}{2}\right)}_{x=0}^{x=1}}\\&\end{align}$$
b): Conditional mean of X given Y \(E(X|Y=1)
We know that:
$$\begin{align} E\left(x\middle| y\right)&=\int_{0}^{1}{xg\left(x\middle| y\right)dx}\\ &E\left(x\middle| y\right)=\int_{0}^{1}{x\frac{1-xy}{1-\frac{y}{2}}dx}\\ &=\frac{1}{1-\frac{y}{2}}{\left[\frac{x^2}{2}-\frac{x^3y}{3}\right]\left(\frac{x^2}{2}-\frac{x^3y}{3}\right)}_{x=0}^{x=1}\\ &=\frac{1}{1-\frac{y}{2}}\left(\frac{1}{2}-\frac{y}{3}\right)\end{align}$$
Therefore:
$$E\left(Xx\middle| Y y=1\right)=\frac{1}{1-\frac{1}{2}}\left(\frac{1}{2}-\frac{1}{3}\right)=\frac{1}{3}$$
To find variance in the continuous case, we would just integrate over the same region with \(x^2\) instead of x and then find the difference between this integral and \(\left[E\left(x\middle| y\right)\right]^2\)
We compute and define conditional expectations, variances, etc., as usual, but with conditional distributions in place of ordinary distributions:
In the discrete case,
\(E(X|Y)=E(X|Y=y)=\sum _{ x }^{ }{ xg(x|y) } \)
\(E({ X }^{ 2 }|Y)=E({ X }^{ 2 }|Y=y)=\sum _{ x }^{ }{ { x }^{ 2 }g(x|y) } \)
\(Var(X|Y)=Var(X|Y=y)=E({ X }^{ 2 }|y)-[E(X|y)]^{ 2 }\)
and in general,
\(E(X|\text{condition})=\sum _{ x }^{ }{ xP(X=x|\text{condition}) } \)
In the continuous case,
\(E(X|y)=E(X|Y=y)=\int _{ }^{ }{ xg(x|y)\partial x } \)
\(E({ X }^{ 2 }|y)=E({ X }^{ 2 }|Y=y)=\int _{ }^{ }{ { x }^{ 2 }g(x|y)\partial x } \)
\(Var(X|y)=Var(X|Y=y)=E({ X }^{ 2 }|y)-[E(X|y)]^{ 2 }\)
and in general,
\((E(X|{ \text{condition} })=\int _{ -\infty }^{ \infty }{ xP(X=x|\text{condition}) } \)
The conditional density (pdf or pmf) of \(X\) given that \(Y = y\) is given by:
\(g(x|y)=\cfrac { f(x,y) }{ { f }_{ Y }(y) } \)
The conditional density of \(Y\) given that \(X=x\) is given by:
\(h(y|x)=\cfrac { f(x,y) }{ { f }_{ X }(x) } \)
Learning Outcome
Topic 3.c: Multivariate Random Variables – Calculate moments for joint, conditional, and marginal random variables.