Limited Time Offer: Save 10% on all 2022 Premium Study Packages with promo code: BLOG10

Explain and perform calculations concerning joint probability functions, probability density functions, and cumulative distribution functions.

Explain and perform calculations concerning joint probability functions, probability density functions, and cumulative distribution functions.

Bivariate Distributions (Joint Probability Distributions)

Sometimes certain events can be defined by the interaction of two measurements. These types of events explained by the interaction of the two variables constitute what we call bivariate distributions.

When put simply, bivariate distribution means the probability that a certain event will occur when there are two independent random variables in a given scenario.

For instance,

A case where you have two bowls, and each is carrying many different types of candies. When you take one cady from each bowl, it gives you two independent random variables, that is, the two different candies. The fact that you are taking one candy from each bowl at the same time means that you have a bivariate distribution when you are calculating the probability of ending up with a particular kind of candy.

Bivariate distribution is also referred to as joint probability distribution and defined as the probability distribution of two random variables, \(X\) and \(Y\), defining the simultaneous behavior between the two random variables.

Definition of a Bivariate Distribution

Let \(X\) and \(Y\) be two random variables defined in a discrete space. Let \(S\) denote the corresponding two-dimensional space of \(X\) and \(Y\), the two discrete random variables.

The probability that \(\text{X} = \text{x}\) and \(\text{Y} = \text{y}\) is denoted by \( \text{f} \left( \text{x},\text{y} \right) \) = \( \text{P} \left( \text{X} = \text{x},\text{Y} = \text{y} \right) \). The function \( \text{f} \left( \text{x},\text{y} \right) \) is called the joint probability mass function (joint pmf) of X and Y. It specifies how the total probability of 1 is divided up amongst the possible values of \( \left( \text{x},\text{y} \right) \) and it gives the joint/bivariate probability distribution of \( \left( \text{X},\text{Y} \right) \).

Discrete Case

For two discrete random variables defined in certain probability space, the probability of a certain event for each random variable is denoted by a joint probability mass function (pmf). Properties of probabilities bound these functions:

Properties of a Discrete Joint/Bivariate Distribution

  1. \(0 \le \text{f} \left( \text{x},\text{y} \right)\le1\); It means that the probability of any possible outcome must not be less than 0 and greater than 1.
  2. \({ \sum }_{ \text{x} }{ \sum }_{ \text{y} } \text{f} \left( \text{x},\text{y} \right)=1,\forall \left( \text{x},\text{y} \right)\epsilon \text{S} \); The condition requires all the probabilities over the entire space \(S\) to sum up to 1.
  3. \( \text{P} \left[\left(\text{X},\text{Y}\epsilon\text{A} \right) \right]={ \sum }_{\text{x},\text{y}\epsilon \text{A}}{ \sum }\text{f} \left( \text{x},\text{y} \right) \); where \(A\) is a subset of the space \(S\). It means that whenever we want to establish the probability of a given event \(A\), we do so by simply summing up the probabilities of the \(\left( \text{x},\text{y} \right)\) values in \(A\).

Example: Bivariate Distribution (Discrete Case) #1

Suppose a certain local bank had three deposit or withdrawal counters. Two clients arrive at the counters at different times when the counters are serving no other customers. Each client chooses a counter at random, independently of the other.

Calculate the joint probability function of \(X\) and \(Y\) if \(X\) denotes the number of clients to select counter one and \(Y\) to represent the number of clients to select counter two.

Solution

First, we have to consider the sample space associated with the experiment. Let the pair \({i,j}\) represent the simple event that the first client selects counter \(i\) and the second client chooses counter \(j\), where \(i,j=1,2, and 3\).

By using the mn rule, the sample space consists of:

$$ 3\times3=9 \text{ sample points} \\ \therefore \text{ each sample point is equal and has a probability of } \cfrac{1}{9} $$

Thus, the sample space to the experiment is given as:

$$ \text{S}=\left[\left\{1,1\right\},\left\{1,2\right\},\left\{1,3\right\},\left\{2,1\right\},\left\{2,2\right\},\left\{2,3\right\},\left\{3,1\right\},\left\{3,2\right\},\left\{3,3\right\}\right] $$

We know that:

$$ \text{f} \left( { \text{x} },{ \text{y} } \right) = \text{P} \left( \text{X} = \text{x},\text{Y} = \text{y} \right) $$

Therefore, the joint probability of \(X\) and \(Y\) is given as follows:

$$ \begin{array}{c|c|c|c} {\quad \text X }& {0} & {1} & {2} \\ {\Huge \diagdown } & & & \\ {\text Y \quad} & & & \\ \hline {0} & {\cfrac{1}{9}} & {\cfrac{2}{9}} & {\cfrac{1}{9}} \\ \hline {1} & {\cfrac{2}{9}} & {\cfrac{2}{9}} & {0} \\ \hline {2} & {\cfrac{1}{9}} & {0} & {0} \end{array} $$

Example: Bivariate Distribution (Discrete Case) #2

Given the table of probabilities above, calculate:

  1. \( \text{P} \left( \text{X}=2,\text{Y}=1 \text{ or } 2 \right)\)
  2. \(\text{P} \left( \text{Y}=2 \right)\)
  3. \(\text{P} \left( \text{X}=0|\text{Y}=1 \right)\)

Solution

  1. We know that,

    $$ \text{f} \left( { \text{x} },{ \text{y} } \right) = \text{P} \left( \text{X} = \text{x},\text{Y} = \text{y} \right) $$

    Thus,

    $$ \begin{align*} \text{P} \left(\text{X}=2,\text{Y}=1 \text{ or } 2 \right)&=\text{P}\left(\text{X}=2,\text{Y}=1\right)+\text{P}\left(\text{X}=2,\text{Y}=2\right) \\ &=0+0=0 \\ \therefore \text{P} \left(\text{X} =2,\text{Y}=1 \text{ or } 2\right) & =0\end{align*} $$

  2. We are required to find \(\text{P} \left(\text{Y}=2 \right)\), and since it does not depend on the value of X, it is the same as finding \( \text{P} \left(\text{Y}=2 ,\text{X}=0,1,2\right) \). That is, we are summing over all the possible values of X.

    Thus,

    $$ \begin{align*} \text{P} \left(\text{Y}=2 \right)&=\cfrac{1}{9}+0+0 \\ \therefore \text{P} \left(\text{Y} =2 \right)& =\cfrac{1}{9} \end{align*} $$

  3. By use of the formula for conditional probability,\( \text{P} \left( \text{A}| \text{B} \right)= \cfrac{\text{P} \left(\text{A}∩\text{B} \right)}{ \text{P} \left( \text{B} \right)} \)

    Thus,

    $$ \text{P} \left(\text{X}=0│\text{Y}=1 \right)= \cfrac{\text{P}\left(\text{X}=0,\text{Y}=1 \right)} {\text{P}\left(\text{Y}=1 \right) } $$

First, let’s find \(\text{P}\left(\text{Y}=1 \right)\), which is the same as finding \( \text{P}\left(\text{Y}=1,\text{X}=0,1,2\right) \).

Thus,

$$ {\text{P}\left(\text{Y}=1 \right) }=\cfrac{2}{9}+\cfrac{2}{9}+0=\cfrac{4}{9} $$

Then,

$$ \text{P} \left(\text{X}=0│\text{Y}=1 \right)=\cfrac{ \cfrac{2}{9} }{ \cfrac{4}{9} }=\cfrac{1}{2} $$

Continuous Case

Joint distributions of two random variables of the discrete type can be extended to two random variables of the continuous type. The definitions are the same, except that the summations are replaced with integrals. The joint probability density function (joint pdf) of two continuous-type random variables is an integrable function \(\text{f} \left( { \text{x} },{ \text{y} } \right)\) with the following properties (the same properties extended to this case):

  1. \( \text{f}\left(\text{x},\text{y}\right)\ge0, \)where \(\text{f} \left( { \text{x} },{ \text{y} } \right)\) is not in the support (space) \(S\) of \(X\) and \(Y\).
  2. \({ \int }_{ -\infty }^{ \infty }{ \int }_{ -\infty }^{ \infty } \text{f}\left(\text{X},\text{Y}\right)\text{dxdy}=1\). Note that the properties above are parallel to those for a single random variable, where the probability density function was \(\text{f} \left( { \text{x} } \right)\) which had to satisfy \(\text{f} \left( { \text{x} } \right) \ge 0\) for all values of x and \( { \int }_{ \text{x} }\text{f} \left( { \text{x} } \right) \text{dx}=1\). Also, remember that probabilities were calculated using: $$ \text{P} \left( \text{a}<{ \text{X} }<{ \text{b} } \right)={ \int }_{ \text{a} }^{ \text{b} }\text{f} \left( { \text{x} } \right)\text{dx} $$
  3. \(\text{P} \left[ \left({ \text{X} }_1,{ \text{X} }_2 \right)\epsilon { \text{A} } \right]={ \int }_{ \text{A} } { \int }\text{f} \left( { \text{x} },{ \text{y} } \right)\text{dxdy},\) where \(\left({ \text{X} }_1,{ \text{X} }_2 \right) \epsilon \text{A} \) is an event defined in the plane. For the case of two continuous variables, the distribution of probability over a specified area in the \((x, y)\) plane is given by the (joint) probability density function \(f(x,y)\). The probability that the pair \(\left( { \text{X} },{ \text{Y} } \right)\) takes values in some specified region \(A\) is obtained by double integrating \(\text{f} \left( { \text{x} },{ \text{y} } \right)\) over \(A\). Thus, $$ \text{p}\left({ \text{x} }_{ 1 }<\text{X}<{ \text{x} }_{ 2 },{ \text{y} }_{ 1 }<\text{Y}<{ \text{y} }_{ 2 }\right)={ \int }_{ { \text{y} }_{ 1 } }^{ { \text{y} }_{ 2 } }{ \int }_{ { \text{x} }_{ 1 } }^{ { \text{x} }_{ 2 } }\text{f} \left( { \text{x} },{ \text{y} } \right)\text{dx}\text{ dy} $$

The joint distribution function \(\text{f} \left( { \text{x} },{ \text{y} } \right)\) is defined by:

$$ \text{f} \left( { \text{x} },{ \text{y} } \right)=\text{P}\left(\text{X}\le\text{x},\text{Y}\le\text{y}\right) $$

And it is related to the joint density function by:

$$ \text{f} \left( { \text{x} },{ \text{y} } \right)= \cfrac{{ \partial }^{ 2 }}{{ \partial }{\text{x}}{ \partial }{\text{y}}}\text{F} \left( { \text{x} },{ \text{y} } \right) $$

Example: Joint Probability Distribution (Continuous Case)

The continuous random variables \(X\) and \(Y\) have the following joint probability density function:

$$
f(x, y)= \begin{cases}2 e^{-2x} e^{-y} & 0<x<\infty, 0<y<\infty \\ 0 & \text { otherwise }\end{cases}
$$

Calculate \( \text{P} \left(X < Y\right) \).

Solution

We have:

$$\begin{align} P(X < Y) &=\int_{(x, y): x < y} {2 e^{-2x} e^{- y}} d x d y \\
&=\int_{0}^{\infty} {\int_{0}^{y} {2 e^{-2x} e^{-y} d x d y}}\\ &= \int_{0}^{\infty}-e^{-y}\left(e^{-2 y}-1\right) d y=\frac{2}{3} \end{align} $$

Marginal Probability Distribution

If we are provided with a joint probability distribution for \(X\) and \(Y\), we can obtain the individual probability distribution for \(X\) or \(Y\), and this is what is referred to as a marginal probability distribution.

Marginal Probability Mass Function

Suppose we are interested in the distribution of one of the two variables without considering the values that the other variable can take. This would give us the marginal probability mass function.

Once we have these marginal distributions, then we can analyze the two variables separately.

Note: If \(X\) and \(Y\) are discrete variables, this distribution can be described with a joint probability mass function.

Recall that the joint probability mass function of the discrete random variables \(X\) and \(Y\) denoted as \( {\text{f}}_{\text{XY}} \left( { \text{x} },{ \text{y} } \right)\), satisfies the following properties:

  1. \({\text{f}}_{\text{XY}} \left( { \text{x} },{ \text{y} } \right)\ge0\)
  2. \({ \sum }_{ \text{x} }{ \sum }_{ \text{y} }{\text{f}}_{\text{XY}} \left( { \text{x} },{ \text{y} } \right)=1\)
  3. \({\text{f}}_{\text{XY}} \left( { \text{x} },{ \text{y} } \right)=\text{P}\left(\text{X}=\text{x},\text{Y}=\text{y}\right)\)

Let \(X\) and \(Y\) have the joint probability mass function \(f(x,y)\) with space \(S\). The probability mass function of \(X\) alone, which is called the marginal probability mass function of \(X\), is defined by:

$$ { \text{f}}_{\text{x}} (\text{x})=\sum _{ y }^{ }{ {\text{f}} \left( { \text{x} },{ \text{y} } \right) } =\text{P}\left(\text{X}=\text{x}\right),\text{x}\epsilon {\text{S}}_{\text{x}} $$

where the summation is taken over all possible values of \(y\) for each given \(x\) in \(x\) space \({\text{S}}_{\text{x}}\).

It implies that the summation is over all \((x,y)ϵS\) with a given value \(x\).

Similarly, the marginal probability mass function of \(Y\) is defined by:

$$ { \text{f}}_{\text{y}} (\text{y})=\sum _{ x }^{ }{ {\text{f}} \left( { \text{x} },{ \text{y} } \right) } =\text{P}\left(\text{Y}=\text{y}\right),\text{y}\epsilon {\text{S}}_{\text{y}} $$

where the summation is taken over all possible values of \(x\) for each given \(y\) in \(y\) space \({\text{S}}_{\text{y}}\).

Example: Marginal Probability Mass Function (Discrete Case) #1

Suppose that the joint pmf of \(X\) and \(Y\) is given as:

$$ {\text{f}} \left( { \text{x} },{ \text{y} } \right) =\cfrac{ {\text{x} }^{2} { \text{y} }^{3}}{50}, \text{x}=1,2,3 \text{ y}=1,2 $$

  1. Calculate the \({\text{f}}_{\text{x}} \left(\text{x}\right)\) (marginal probability mass function of X)
  2. Calculate \({\text{f}}_{\text{y}} \left(\text{y}\right)\) (marginal probability mass function of Y)

Solution

  1. We know that \({\text{f}}_{\text{x}} \left(\text{x}\right)\) is given as

    $$ {\text{f}}_{\text{x}} \left(\text{x}\right)=\sum _{ y }^{ }{ {\text{f}} \left( { \text{x} },{ \text{y} } \right) }=\text{P}\left(\text{X}=\text{x}\right),\text{x}\epsilon {\text{S}}_{\text{x}} $$

    Thus,

    $$ \begin{align*} {\text{f}}_{\text{x}} \left(\text{x}\right)&=\sum _{ y=1 }^{ 2 }=\cfrac{{ \text{x} }^{ 2 }{ \text{y} }^{ 3 }}{50} \quad \text{y}=1,2 \\ &=\cfrac{{ \text{x} }^{ 2 }{ \left( 1 \right) }^{ 3 }}{50}+\cfrac{{ \text{x} }^{ 2 }{ \left( 2 \right) }^{ 3 }}{50} \\ \therefore { \text{f} }_{ \text{x} } \left(\text{x}\right) & =\cfrac{{9 { \text{x} } }^{2}}{50} \end{align*} $$

  2. We know that,

    $$ {\text{f}}_{\text{y}} \left(\text{y}\right)=\sum _{ x }^{ }{ {\text{f}} \left( { \text{x} },{ \text{y} } \right) }=\text{P}\left(\text{Y}=\text{y}\right),\text{y}\epsilon {\text{S}}_{\text{y}} $$

    Thus,

    $$ \begin{align*} {\text{f}}_{\text{y}} \left(\text{y}\right)&=\sum _{ x=1 }^{ 3 }=\cfrac{{ \text{X} }^{ 2 }{ \text{y} }^{ 3 }}{50}, \text{ x}=1,2,3 \\ &=\cfrac{{ \left( 1 \right) }^{ 2 }{ \text{y} }^{ 3 }}{50}+\cfrac{{ \left( 2 \right) }^{ 2 }{ \text{y} }^{ 3 }}{50}+\cfrac{{ \left( 3 \right) }^{ 2 }{ \text{y} }^{ 3 }}{50} \\ &=\cfrac{1{\text{y}}^{3}}{50}+\cfrac{4{ \text{y} }^{3}}{50}+\cfrac{9{ \text{y} }^{3}}{50} \\ \therefore { \text{f} }_{ \text{y} } \left(\text{y}\right) & =\cfrac{{14 { \text{y} } }^{3}}{50}=\cfrac{7}{25} { \text{y} }^{3} \end{align*} $$

Example: Marginal Probability Mass Function (Discrete Case) #2

Suppose \(X\) and \(Y\) have the joint probability function as given in the table below:

$$ \begin{array}{c|c|c|c} {\quad \text X }& {1} & {2} & {3} \\ {\Huge \diagdown } & & & \\ {\text Y \quad} & & & \\ \hline {1} & {0.12} & {0.15} & {0.03} \\ \hline {2} & {0.13} & {0.05} & {0.07} \\ \hline {3} & {0.18} & {0.06} & {-} \\ \hline {4} & {0.17} & {0.04} & {-} \end{array} $$

Calculate the probability distribution of \(X\).

We know that:

$$ { \text{f}}_{\text{x}} (\text{x})=\sum _{ y }^{ }{ {\text{f}} \left( { \text{x} },{ \text{y} } \right) } =\text{P}\left(\text{X}=\text{x}\right),\text{x}\epsilon {\text{S}}_{\text{x}} $$

So that the marginal probabilities are:

$$ \begin{align*} \text{P}\left(\text{X}=1\right)&=0.12+0.13+0.18+0.17=0.6 \\ \text{P}\left(\text{X}=2\right)&=0.15+0.05+0.06+0.04=0.3 \\ \text{P}\left(\text{X}=3\right)&=0.03+0.07=0.1 \end{align*} $$

Therefore, the probability distribution of \(X\) is:

$$ \begin{array}{c|c|c|c} \textbf{X} & \textbf{1} & \textbf{2} & \textbf{3} \\ \hline {\text{P}\left(\text{X}=\text{x}\right)} & {0.6} & {0.3} & {0.01} \end{array} $$

Marginal Probability (Density) Function

In the case of continuous variables, the marginal probability density function (pdf) of \(X\), \({ \text{f}}_{\text{x}} \left(\text{x}\right)\), is obtained by integrating over \(y\) (for the given value of \(x\)) the joint pdf \( \text{f} \left( { \text{x} },{ \text{y} } \right) \). The same case goes for marginal probability density function (PDF) of Y, \({ \text{f}}_{\text{y}} \left(\text{y} \right)\), where we obtain it by integrating over \(x\) (for the given value of \(y\)).

The respective marginal pdfs of continuous-type random variables \(X\) and \(Y\) are given by:

$$ { \text{f}}_{\text{x}} \left(\text{x}\right)={ \int }_{ -\infty }^{\infty}{\text{f}} \left( { \text{x} },{ \text{y} } \right) \text{dy}, \quad \text{x}\epsilon{\text{S}}_{\text{x}}, $$

And,

$$ { \text{f}}_{\text{y}} \left(\text{y}\right)={ \int }_{ -\infty }^{\infty}{\text{f}} \left( { \text{x} },{ \text{y} } \right) \text{dx}, \quad \text{y}\epsilon{\text{S}}_{\text{y}}, $$

The resulting \({ \text{f}}_{\text{x}} \left(\text{x}\right)\) and \({ \text{f}}_{\text{y}} \left(\text{y}\right)\) is a proper pdf, meaning it integrates to 1 where \({\text{S}}_{\text{x}}\) and \({\text{S}}_{\text{y}}\) are the respective spaces of \(X\) and \(Y\). The definitions associated with the mathematical expectations in the continuous case are the same as those associated with the discrete case after replacing the summations with the integrations. Recall that if \(X\) and \(Y\) are continuous random variables, this distribution can be described with a joint probability density function which satisfies the following properties:

  1. \({\text{f}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right)\ge 0 \) for all \(x,y\).
  2. \({ \int }_{ -\infty }^{ \infty }{ \int }_{ -\infty }^{ \infty }{\text{f}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right)\text{dx} \text{ dy}=1 \).
  3. For any space R, $$ \text{P}\left(\left(\text{X},\text{Y}\right)\epsilon \text{R}\right)={\int }{\int }_{\text{R}} {\text{f}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right)\text{dx} \text{ dy} $$ when the random variables are continuous.

Example: Marginal Probability (Density) Function

Given that \(X\) and \(Y\) have the following joint pdf:

$$ \text{f} \left( { \text{x} },{ \text{y} } \right)= \left( \cfrac{8}{9}\right) \left( 1+ \text{xy} \right), \quad 0\le\text{x}\le1, 0\le\text{y}\le1 $$

Calculate the marginal pdf of \(X\) and \(Y\).

Solution

We know that:

$$ { \text{f}}_{\text{x}} \left(\text{x}\right) ={ \int }_{ -\infty }^{ \infty }{\text{f}} \left( { \text{x} },{ \text{y} } \right)\text{dy}, \quad \forall \text{x}\epsilon{\text{S}}_{\text{x}} $$

Thus,

$$ \begin{align*} { \text{f}}_{\text{x}} \left(\text{x}\right) &= { \int }_{ 0 }^{ 1 }\left( \cfrac{8}{9}\right) \left( 1+ \text{xy} \right)\text{dy} \\ &=\cfrac{8}{9}\left({ \int }_{ 0 }^{ 1 }1 \text{dy} + \text{x}{ \int }_{ 0 }^{ 1 }\text{y}\text{dy}\right) \\ \therefore { \text{f}}_{\text{X}}\left(\text{x}\right) & =\cfrac{8}{9}\left(1+ \cfrac {\text {x}}{2}\right), \quad 0\le\text{x}\le 1 \end{align*} $$

And,

We know that:

$$ { \text{f}}_{\text{y}} \left(\text{y}\right) ={ \int }_{ -\infty }^{ \infty }{\text{f}} \left( { \text{x} },{ \text{y} } \right)\text{dx}, \quad \text{y}\epsilon{\text{S}}_{\text{y}} $$

Thus,

$$ \begin{align*} { \text{f}}_{\text{y}} \left(\text{y}\right) &= { \int }_{ 0 }^{ 1 }\left( \cfrac{8}{9}\right) \left( 1+ \text{xy} \right)\text{dx} \\ &=\cfrac{8}{9}\left[\int _{ 0 }^{ 1 }{ 1 } \text{dx}+\text y\int _{ 0 }^{ 1 }{ \text x } {\text{dx}} \right] \\ \therefore { \text{f}}_{\text{y}} \left(\text{y}\right) & = \cfrac{8}{9}\left(1+ \cfrac{\text y}{2}\right), 0\le\text{y}\le 1 \end{align*} $$

Independent Random Variables

\(X\) and \(Y\) random variables are independent if, and only if, the joint probability function is the product of the two marginal probability functions for all \(x,y\) in the range of the variables, that is:

$$ {\text{f}}_{\text{X},\text{Y}} \left( { \text{x} },{ \text{y} } \right)={ \text{f}}_{\text{x}} \left(\text{x}\right) { \text{f}}_{\text{y}} \left(\text{y}\right) \text{for all } \left( { \text{x} },{ \text{y} } \right) \text{ in the range} $$

Discrete Case

For discrete independent random variables, they go by the probability statement concerning values assumed by \(X\), \(Y\) where they can be broken down into statements relating to \(X\) and \(Y\) separately.

Thus, if \(X\) and \(Y\) are independent discrete variables then:

$$ \text{P}\left(\text{X}=\text{x},\text{Y}=\text{y} \right)=\text{P}\left(\text{X}=\text{x}\right)\text{P}\left(\text{Y}=\text{x}\right) $$

For instance, let:

$$ {\text{f}} \left( { \text{x} },{ \text{y} } \right) =\cfrac{ {\text{x} }^{2} { \text{y} }^{3}}{50}, \text{x}=1,2,3 \quad \text{ y}=1,2 $$

So that \( { \text{f}}_{\text{x}} \left(\text{x}\right)=\cfrac{9 \text{x}^{2}}{50} \text{ and } { \text{f}}_{\text{y}} \left(\text{y}\right)=\cfrac{14 \text{y}^{3}}{50} \).

In that, in this case,

$${\text{f}} \left( { \text{x} },{ \text{y} } \right)\neq { \text{f}}_{\text{X}} \left(\text{x}\right) { \text{f}}_{\text{Y}} \left(\text{y}\right) \ \text{for} \ \text{x}=1,2,3 \text{ and } \text{y}=1,2  $$

And thus \(X\) and \(Y\) are dependent.

Example: Independent Random Variables (Discrete Case)

Suppose \(X\) and \(Y\) have the joint probability function as given in the table below;

$$ \begin{array}{c|c|c|c} {\quad \text X }& {1} & {2} & {3} \\ {\Huge \diagdown } & & & \\ {\text Y \quad} & & & \\ \hline {1} & {0.12} & {0.15} & {0.03} \\ \hline {2} & {0.13} & {0.05} & {0.07} \\ \hline {3} & {0.18} & {0.06} & {0} \\ \hline {4} & {0.17} & {0.04} & {0} \end{array} $$

Determine whether the variables \(X\) and \(Y\) are independent.

Solution

First,

$$ \text{P}\left(\text{X}=1,\text{Y}=1\right)=0.12 $$

However,

$$ \text{P}\left(\text{X}=1\right)=\left(0.12+0.13+0.18+0.17\right)=0.6 $$

And,

$$ \text{P}\left(\text{Y}=1\right)=\left(0.12+0.15+0.03\right)=0.3 $$

Thus,

$$ 0.6\times0.3 \neq 0.12 $$

Therefore, the two random variables are NOT independent.

Note: In order to show that random variables are not independent, we are only required to show that the joint probability is not equal to the product of the marginal probabilities in any of a particular case. Otherwise, they are independent.

Continuous Case

Let \(X\) and \(Y\) be continuous independent random variables with a double integral required for evaluating the joint probability, in which it splits into the multiplication of two separate integrals, one for \(X\) and one for \(Y\), resulting into:

$$ \text{P}\left({ \text{x} }_{1}< { \text{X} }< { \text{x} }_{2},{ \text{y} }_{1}<{ \text{Y} }<{ \text{y} }_{2} \right)=\text{P}\left({ \text{x} }_{1}<\text{X}<{ \text{x} }_{2}\right)\text{P}\left({ \text{y} }_{1}<{ \text{Y} }<{ \text{y} }_{2}\right) $$

In that, it simply means that for the continuous case, where the two random variables are independent, we can factorize the joint pdf into two separate expressions, in which one of the functions will be of \(x\) only, and the other will be a function of \(y\) only.

Example: Independent Continuous Random Variables

The continuous random variables \(X\) and \(Y\) have the following joint probability density function:

$$ {\text{f}} \left( { \text{x} },{ \text{y} } \right)=\cfrac{8}{9} \left(1+ {\text{xy}} \right), \quad 0\le{ \text{x} }\le1, \quad 0\le{ \text{y} }\le1 $$

Determine whether \(X\) and \(Y\) are independent.

Solution

Recall that:

$$ { \text{f}}_{\text{x}} \left(\text{x}\right)=\cfrac{8}{9}\left(1+ { \cfrac{ \text{x} }{2}} \right), \quad 0\le{ \text{x} }\le 1 $$

And

$$ { \text{f}}_{\text{y}} \left(\text{y}\right)=\cfrac{8}{9}\left(1+ { \cfrac{ \text{y} }{2}} \right), \quad 0\le{ \text{y} }\le1 $$

It is clear to see that \( {\text{f}} \left( { \text{x} },{ \text{y} } \right)\neq { \text{f}}\left(\text{x}\right) { \text{f}}\left(\text{y}\right) \) and hence the variables \(X\) and \(Y\) are not independent.

Expectations of Functions of Two variables

Discrete Variables

Sometimes, we may encounter multivariate (more than two) functions. For this reason, it is important to switch notation from \({ \text{X} } \) and \({ \text{Y} } \) to \({ \text{X} }_1\) and \({ \text{X} }_2\). However, the \( ({ \text{X} } \) and \({ \text{Y} } \) notation will be used on integrals, and we are simply assuming that \( { \text{X} }_{1}={ \text{x} }\) and \( { \text{X} }_{2}={ \text{y} } \) ).

The expected value of the function \( \text{u} \left( { \text{X} }_{1},{ \text{X} }_{2} \right) \) of the random variables \( \left( { \text{X} }_{1},{ \text{X} }_{2} \right) \) can be found by summing the product:

\(\text{Value} × \text{Probability of assuming that value}\) over all the values (or combinations) of \( \left( { \text{x} }_{1},{ \text{x} }_{2} \right) \)

When dealing with discrete random variables, therefore:

$$ \text{E}\left[\text{u}({ \text{X} }_{1},{ \text{X} }_{2} )\right]=\sum _{ \left( { \text{x} }_{ 1 },{ \text{x} }_{ 2 } \right) \epsilon \text{S} }^{ }{ \sum { \text{u}\left( { \text{x} }_{ 1 },{ \text{x} }_{ 2 } \right) \text{f}\left( { \text{x} }_{ 1 },{ \text{x} }_{ 2 } \right) } } $$

Where the double summation is over all possible values of \({ \text{x} }_{ 1 }\) and \({ \text{x} }_{ 2 }\).

Example: Expectations of Functions of Two Variables (Discrete Case)

\({ \text{X} }_{ 1 }\) and \({ \text{X} }_{ 2 }\) have the following joint pmf:

$$ \text{f}\left({ \text{X} }_{ 1 },{ \text{X} }_{ 2 } \right)=\cfrac { { \text{x} }_{ 1 }{ \text{x} }_{ 2 }^{ 2 } }{ 50 } \text{ where } { \text{x} }_{ 1 }=1,2,3 \text{ and } { \text{x} }_{ 2 }=1,2 $$

Calculate the expected value of \( \left[{ \text{X} }_{ 1 }+{ \text{X} }_{ 2 }\right] \).

Solution

We know that:

$$ \text{E}\left[\text{u}({ \text{X} }_{1},{ \text{X} }_{2} )\right]=\sum _{ \left( { \text{x} }_{ 1 },{ \text{x} }_{ 2 } \right) \epsilon \text{S} }^{ }{ \sum { \text{u}\left( { \text{x} }_{ 1 },{ \text{x} }_{ 2 } \right) \text{f}\left( { \text{x} }_{ 1 },{ \text{x} }_{ 2 } \right) } } $$

Thus,

$$ \begin{align*} \text{E}\left[{ \text{X} }_{1}+{ \text{X} }_{2}\right]&= \sum _{ { \text{x} }_{ 1 }=1 }^{ 3 }{ \sum _{ { \text{x} }_{ 2 }=1 }^{ 2 }{ \left({ \text{x} }_{ 1 }+{ \text{x} }_{ 2 } \right)\text{P}\left({ \text{X} }_{ 1 }={ \text{x} }_{ 1 },{ \text{X} }_{ 2 }={ \text{x} }_{ 2 } \right) } } \\ \text{E}\left[{ \text{X} }_{1}+{ \text{X} }_{2}\right]&=\sum _{ { \text{x} }_{ 1 }=1 }^{ 3 }{ \sum _{ { \text{x} }_{ 2 }=1 }^{ 2 }{ \left({ \text{x} }_{ 1 }+{ \text{x} }_{ 2 } \right) \left(\cfrac {{{ \text{x} }_{ 1 } { \text{x} }_{ 2 }^{2} }}{{30}} \right) } } \\ \text{E}\left[{ \text{X} }_{1}+{ \text{X} }_{2}\right]&=\sum _{ { \text{x} }_{ 2 }=1 }^{ 2 }\left(1+{ \text{X} }_{2} \right)\cfrac{1{ \text{x} }_{ 2 }^{2}}{30}+\left(2+{ \text{x} }_{ 2 } \right)\cfrac{2{ \text{x} }_{ 2 }^{2} }{30}+\left(3+{ \text{x} }_{ 2 } \right)\cfrac{3{ \text{x} }_{ 2 }^{2}}{30} \\ \text{E}\left[{ \text{X} }_{1}+{ \text{X} }_{2}\right]&= \left(2\right) \cfrac{1}{30} +\left(3\right) \cfrac{2}{30} +\left(4\right) \cfrac{3}{30} +\left(3\right) \cfrac{4}{30} +\left(4\right) \cfrac{8}{30} +\left(5\right) \cfrac{12}{30} =\cfrac{124}{30} \end{align*} $$

Continuous Variables

The expectation of continuous variables, say \(X\) and \(Y\), is given by:

$$E\left[g\left(X,Y\right)\right]=\int_{x}\int_{y}{g\left(x,y\right)f_{XY}\left(x,y\right)dydx}$$

Where the integration is over all possible values of \(x\) and \(y\).

This result parallels that for single random variables, where the expected value of a function of a continuous random variable was defined to be:

$$ \text{E}\left[\text{g}\left(\text{X}\right)\right]={ \int }_{ \text{x} }^{ }\text{g}\left(\text{X}\right)\text{f}\left(\text{x}\right)\text{dx} $$

Given a joint probability distribution, sometimes you may be required to determine values related to only one of the two variables, say \(\text{E}\left(\text{X}\right)\) or \(\text{V}\left(\text{X}\right)\).

To do so, first, calculate the marginal distribution of the relevant variable, say X, and then proceed to work out the solution as we would normally do for the univariate case.

Example: Expected Value of a Continuous Random Variable (Joint Probability Distribution)

The joint density of \(X\) and \(Y\) is given by:

$$ {\text{f} }_{ \text{XY} }\left( { \text{x} },{ \text{y} } \right)=\cfrac {2 \text{x+y} }{3000} \text{ where } 10<\text{x} < 20 \text{ and} -5 < \text{y} < 5 $$

  1. Find \(E\left( \text{X} \right)\).Solution

    First, we find the PDF of the marginal distribution of X, by integrating out \(Y\). We know that:

    $$ { \text{f}}_{\text{x}} \left(\text{x}\right)={ \int }_{ \infty }^{ \infty }{\text{f}} \left( { \text{x} },{ \text{y} } \right)\text{dy}, \quad \text{x}\epsilon{\text{S}}_{\text{x}}, $$

    Thus, the marginal distribution of X is:

    $$ \begin{align*} { \text{f}}_{\text{x}} \left(\text{x}\right)&={ \int }_{ -5 }^{ 5 }\cfrac {2 \text{x+y} }{3000}{ \partial }{ \text{y}} \\ &=\cfrac{1}{3000}\left| 2 \text{xy} + \cfrac{ { \text{y} }^{2} }{2} \right|_{ -5 }^{ 5 } \\ &=\cfrac{1}{3000} \left\{ \left( 10 { \text{x} }+ \cfrac{ 25 }{2} \right) -\left(-10 { \text{x} }+ \cfrac{ 25 }{2} \right) \right\} \\ \therefore { \text{f}}_{\text{x}} \left(\text{x}\right) & =\cfrac{ \text{x} }{ 150 }, \quad 10 < {\text{x}} < 20 \end{align*} $$

    Then, we know that:

    $$ { \mu }_{ \text{X} }={ \text{E} }\left({ \text{X} } \right)={ \int }_{ – \infty }^{ \infty }\text{x} { \text{f} }_{ \text{X} } \left(\text{x}\right) \text{dx} $$

    Thus,

    $$ \begin{align*} { \text{E} }\left({ \text{X} } \right)&={ \int }_{ 10 }^{ 20 }\text{x} \cfrac{ \text{x} }{150}{ \partial }{ \text{x} }\\ &=\cfrac{ 1 }{150} \left| \cfrac{{ \text{x} }^{3} }{3} \right|_{10}^{20} \\ &=\cfrac{ 1 }{150} \left( \cfrac{8,000}{3}- \cfrac{1,000}{3} \right) \\ \therefore { \text{E} }\left({ \text{X} } \right) & = \cfrac{7,000}{450}=\cfrac{140}{9} \end{align*} $$

  2. Find the variance of \(X\).Solution

    We know that,

    $$ \begin{align*} { \sigma }_{ \text{x} }^{2}&={ \text{E} }\left({\text{X}}^{2} \right)-\left[\text E\left( \text{X} \right)\right]^{2} \\ &=\left({ \int }_{ – \infty }^{ \infty }{ \text{x} }^{2}.{ \text{f} }\left(\text{x}\right)\text{dx} \right)-{ \mu }^{2} \end{align*} $$

    Thus,

    $$ \begin{align*} { \text{E} }\left({\text{X}}^{2} \right)& ={ \int }_{ 10 }^{ 20 }{\text{x}}^{2}\cfrac{ \text{x} }{150}{ \partial }{ \text{x} } \\ &=\cfrac{ 1 }{150} \left| \cfrac{{ \text{x} }^{4}}{4} \right|_{10}^{20} \\ &=\cfrac{ 1 }{150} \left( \cfrac{160,000}{4}- \cfrac{10,000}{4} \right) \\ \therefore { \text{E} }\left({ \text{X} }^{2} \right) & =\cfrac{150,000}{4}\times \frac{1}{150} \end{align*} $$

    Therefore, we have:

    $$ { \text{E} }\left({ \text{X} }^{2} \right)=250 $$

    And,

    $$ { \text{E} }\left({ \text{X} } \right)=\cfrac{140}{9} $$

    We know that:

    $$ Var(X)={σ}_{x}^{2} \left( \text{Var} \left({ \text{X} } \right) \right)={ \text{E} }\left({ \text{X} }^{2} \right)-\left[{ \text{E} }\left({ \text{X} }\right) \right]^{2} $$

    Then,

    $$ \text{Var} \left({ \text{X} } \right)=\cfrac{150,000}{4}-\left(\cfrac{140}{9}\right)^{2}=\cfrac{3,017,900}{81} $$

Joint Cumulative Distribution Functions

Definition

The joint cumulative distribution function, \({\text{F}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right)\), of two random variables, \(X\) and \(Y\) is defined as the probability that the random variable \(X\) is less than or equal to a specified value of \(x\) and that the random variable Y is less than or equal to a specified value of \(y\).

$$ {\text{F}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right)=\text{P}\left( \text{X}\le\text{x},\text{Y}\le\text{y} \right) $$

Therefore, the Marginal cumulative distribution functions of \(X\) and \(Y\) are:

$$ \begin{align*} { \text{F}}_{\text{x}} \left(\text{x}\right)&={\text{F}}_{ \text{XY} } \left( { \text{x} },{ \infty } \right)=\underset { \text{y} \rightarrow \infty }{ \text{lim} } ⁡{\text{F}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right), \text{for any x}, \\ { \text{F}}_{\text{y}} \left(\text{y}\right)&={\text{F}}_{ \text{XY} } \left( { \infty },{ \text{y} } \right)=\underset { \text{x} \rightarrow \infty }{ \text{lim} } ⁡{\text{F}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right), \text{for any y.} \end{align*} $$

Discrete Random Variables

The joint CDF is given by:

$$ \text{F} \left({ \text{X} }_{1},{ \text{X} }_{2},…,{ \text{X} }_{ \text{n} } \right)= \sum _{ { \text{w} }_{ 1 }\le { \text{x} }_{ 1 } }^{ }{ \sum _{ { \text{w} }_{ 2 }\le { \text{x} }_{ 2 } }^{ }{ \cdots } } \sum _{ { \text{w} }_{ \text{n} }\le { \text{x} }_{ \text{n} } }^{ }{ \text{f}\left({ \text{w} }_{ \text{1} },{ \text{w} }_{ \text{2} },….,{ \text{w} }_{ \text{n} }\right) } $$

Example: Joint CDF for Discrete Random Variables

Consider two random variables, \(X\) and \(Y\), with a joint PMF provided in the table below:

$$ \begin{array}{c|c|c|c} {\quad \text X }& {0} & {1} & {2} \\ {\Huge \diagdown } & & & \\ {\text Y \quad} & & & \\ \hline \textbf{0} & \cfrac{1}{8} & \cfrac{1}{6} & \cfrac{1}{4} \\ \hline \textbf{1} & \cfrac{1}{6} & \cfrac{1}{8} & \cfrac{1}{6} \end{array} $$

Find \({ \text{F} }_{ \text{XY} }\) (0.5,1).

Solution

$$ \text{P}\left({ \text{X} }\le0.5,{ \text{Y} }\le1\right) $$

Thus,

$$ \begin{align*} { \text{P} }_{ \text{XY} } \left(0,0 \right)+ {\text{P} }_{\text{XY}} \left(0,1\right)&=\cfrac{1}{8}+\cfrac{1}{6}=\cfrac{7}{24} \\ \therefore { \text{F} }_{ \text{XY} } \left(0.5,1\right) & =\cfrac{7}{24} \end{align*} $$

Continuous Random Variables

For continuous random variables, the joint CDF is given by:

$$ \text{F} \left({ \text{x} }_{1},{ \text{x} }_{2},…,{ \text{x} }_{ \text{n} } \right)={ \int }_{ -\infty }^{ { \text{x} }_{1} }{ \int }_{ -\infty }^{ { \text{x} }_{2} } \cdots { \int }_{ -\infty }^{ { \text{x } }_{ \text{n} } }\text{f}\left({ \text{w} }_{ \text{1} },{ \text{w} }_{ \text{2} },….{ \text{w} }_{ \text{n} }\right){ \text{dw} }_{ \text{n} }\dots { \text{dw} }_{ \text{2} }{ \text{dw} }_{ \text{1} } $$

Additionally, when \( { \text{X} }_{1},{ \text{X} }_{2},…,{ \text{X} }_{ \text{n} }\) are continuous,

$$ \text{F} \left({ \text{x} }_{1},{ \text{x} }_{2},…,{ \text{x} }_{ \text{n} } \right)=\cfrac{{ \partial } { \text{F} }\left({ \text{x} }_{1},{ \text{x} }_{2},…,{ \text{x} }_{ \text{n} } \right)}{{ \partial }{ \text{x} }_{1}{ \partial }{ \text{x} }_{2}\dots{ \partial }{ \text{x} }_{ \text{n} }} $$

The joint CDF satisfies the following properties:

  1. \({ \text{F}}_{\text{x}} \left(\text{x}\right)={\text{F}}_{ \text{XY} } \left( { \text{x} },{ \infty } \right), \text{ for any x (marginal CDF of X) }\).
  2. \( { \text{F}}_{\text{Y}} \left(\text{y}\right)={\text{F}}_{ \text{XY} } \left({ \infty },{ \text{y} }\right), \text{ for any y(marginal CDF of Y) }\).
  3. \({\text{F}}_{ \text{XY} } \left( { \infty },{ \infty } \right)=1\).
  4. \({\text{F}}_{ \text{XY} } \left({- \infty },{ \text{y} } \right)={\text{F}}_{ \text{XY} } \left({ \text{x} },{- \infty }\right)=0 \)
  5. \(\text{P}\left({ \text{x} }_{1} < \text{X}≤{ \text{x} }_{2},{ \text{y} }_{1}<\text{Y}≤{ \text{y} }_{2} \right) \\ ={\text{F}}_{ \text{XY} } \left({ \text{x} }_{2},{ \text{y} }_{2} \right)-{\text{F}}_{ \text{XY} } \left({ \text{x} }_{1},{ \text{y} }_{2} \right)-{\text{F}}_{ \text{XY} } \left({ \text{x} }_{2},{ \text{y} }_{1} \right)+{\text{F}}_{ \text{XY} } \left({ \text{x} }_{1},{ \text{y} }_{1} \right) \).
  6. If \(X\) and \(Y\) are independent, then \( {\text{F}}_{ \text{XY} }\left( { \text{x} },{ \text{y} } \right)={ \text{f}}_{\text{X}} \left(\text{x}\right) { \text{f}}_{\text{Y}} \left(\text{x}\right) \).

Example: Joint CDF for Continuous Random Variables

Consider the following PDF:

$$ {\text{f}}_{ \text{XY} }\left( { \text{x} },{ \text{y} } \right)=\begin{cases} \text{x} +\cfrac { 3 }{ 2 } \text{y}^{ 2 }\quad \quad 0≤\text{x},\text{y} \le 1 \\ 0,\quad \quad \quad \quad \quad \quad \text{otherwise} \end{cases} $$

Find the joint CDF.

Solution

We know that the joint CDF is given by:

$$ \begin{align*} {\text{F}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right)&={ \int }_{ -\infty }^{ \text{y} }{ \int }_{ -\infty }^{ \text{x} }{\text{f}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right)\text{dxdy} \\ & = { \int }_{ 0 }^{ \text{y} }{ \int }_{ 0 }^{ \text{x} }{\text{f}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right)\text{dxdy} \end{align*} $$

Let \( \text{x}=\text{u} \text{ and } \text{y}=\text{v} \).

Thus,

$$ { \int }_{ 0 }^{ \text{min} \left(\text{y},1 \right) }{ \int }_{ 0 }^{ \text{min}⁡ \left(\text{x},1\right) }\left(\text{u} + \cfrac{3}{2} { \text{v} }^2 \right)\text{dudv} $$

Therefore, for \( 0\le {\text{x} }, { \text{y} }\le1 \) , we obtain:

$$ \begin{align*} {\text{F}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right)&={ \int }_{ 0 }^{ \text{y} }{ \int }_{ 0 }^{ \text{x} }\left(\text{u} + \cfrac{3}{2} { \text{v} }^{2} \right)\text{dudv} \\ & = { \int }_{ 0 }^{ \text{y} } \left[\cfrac{1}{2}{ \text{u} }^{2} + \cfrac{3}{2} { \text{v} }^{2} { \text{u} }\right]_{0}^{ \text{x}}\text{dv} \\ & = { \int }_{ 0 }^{ \text{y} } \left(\cfrac{1}{2}{ \text{x} }^{2} + \cfrac{3}{2} { \text{x} } { \text{v} }^{2} \right)\text{dv} \\ & =\left[\cfrac{1}{2}{ \text{x} }^{2} + \cfrac{3}{2} { \text{x} } { \text{v} }^{2} \right]_{ 0 }^{ \text{y} }=\cfrac{1}{2} { \text{x} }^{2} \text{y}+\cfrac{1}{2} { \text{xy} }^{3} \\ \therefore {\text{F}}_{ \text{XY} } \left( { \text{x} },{ \text{y} } \right) & =\cfrac{1}{2} { \text{x} }^{2} y+\cfrac{1}{2} { \text{xy} }^{3} \end{align*} $$

Learning Outcome

Topic 3.a: Multivariate Random Variables – Explain and perform calculations concerning joint probability functions, probability density functions, and cumulative distribution functions.

Shop CFA® Exam Prep

Offered by AnalystPrep

Featured Shop FRM® Exam Prep Learn with Us

    Subscribe to our newsletter and keep up with the latest and greatest tips for success
    Shop Actuarial Exams Prep Shop GMAT® Exam Prep


    Daniel Glyn
    Daniel Glyn
    2021-03-24
    I have finished my FRM1 thanks to AnalystPrep. And now using AnalystPrep for my FRM2 preparation. Professor Forjan is brilliant. He gives such good explanations and analogies. And more than anything makes learning fun. A big thank you to Analystprep and Professor Forjan. 5 stars all the way!
    michael walshe
    michael walshe
    2021-03-18
    Professor James' videos are excellent for understanding the underlying theories behind financial engineering / financial analysis. The AnalystPrep videos were better than any of the others that I searched through on YouTube for providing a clear explanation of some concepts, such as Portfolio theory, CAPM, and Arbitrage Pricing theory. Watching these cleared up many of the unclarities I had in my head. Highly recommended.
    Nyka Smith
    Nyka Smith
    2021-02-18
    Every concept is very well explained by Nilay Arun. kudos to you man!
    Badr Moubile
    Badr Moubile
    2021-02-13
    Very helpfull!
    Agustin Olcese
    Agustin Olcese
    2021-01-27
    Excellent explantions, very clear!
    Jaak Jay
    Jaak Jay
    2021-01-14
    Awesome content, kudos to Prof.James Frojan
    sindhushree reddy
    sindhushree reddy
    2021-01-07
    Crisp and short ppt of Frm chapters and great explanation with examples.