Limited Time Offer: Save 10% on all 2022 Premium Study Packages with promo code: BLOG10

Determine conditional and marginal probability functions

Determine conditional and marginal probability functions

Conditional Distributions

Conditional probability is a key part of Baye’s theorem, which describes the probability of an event based on prior knowledge of conditions that might be related to the event. It differs from joint probability, which does not rely on prior knowledge.

Example: Baye’s Theorem #1

For instance assume that a law enforcement department is looking into the connection between road accidents and alcohol consumption among automobile drivers. On one hand, the department could develop the probability that a driver is intoxicated and involved in an accident. That would be a joint probability.

On the other hand, the department could determine the probability that a driver is involved in an accident given that they are intoxicated. This would be a conditional probability. The probabilities of these two events cannot be the same. Adding information (in the conditional case) alters probability.

Example: Baye’s Theorem #2

In the medical field, a team of researchers could gather data to help them analyze and predict cases of kidney failure. On one hand, the probability that a patient’s left and right kidneys are both infected is a joint probability.

In contrast, a conditional probability is a probability that the left kidney is infected if we know that the right one is infected.

We can use an Euler diagram to demonstrate the difference between conditional and joint probabilities. In the diagram, each large square has an area 1, and the smaller squares represent probabilities.

Let \(X\) be the probability that a patient’s left kidney is infected, and let \(Y\) be the probability that the right kidney is infected. On the left side of the diagram, the green area represents the probability that both of the patient’s kidneys are affected. This is the joint probability \((X, Y)\). If \(Y\) is true (e.g., given that the right kidney is definitely infected), then the space of everything not \(Y\) is dropped, and everything in \(Y\) is rescaled to the size of the original space.

Baye's TheoremThe rescaled green area on the right-hand side is now the conditional probability of \(X\) given \(Y\), expressed as \(P(X|Y)\). Put differently, this is the probability that the left kidney is infected if we know that the right kidney is infected. It’s also important to note that the conditional probability of \(X\) given \(Y\) is not necessarily equal to the conditional probability of \(Y\) given \(X\).

In simple terms, we define conditional distribution as the distribution of one random variable given the value of another random variable.

Discrete Conditional Functions

The conditional probability mass function of \(X\), given that \(Y = y\), is defined by:

$$ g(x|y) = \frac{f(x,y)}{f_Y(y)}, \qquad \text{provided that } f_Y(y) > 0 $$

Similarly, the conditional probability mass function of \(Y\), given that \(X = x\), is defined by:

$$ h(y|x) = \frac{f(x,y)}{f_X(x)}, \qquad \text{provided that } f_X(x) > 0. $$

If we compare this definition with the univariate case, we had:

$$ P(A|B) = \frac{P(A \cap B)}{P(B)}, \qquad \text{provided that } P(B) > 0. $$

where \(P(B)\) happened first and had an impact on how \(A\) occurred, because if their intersection is empty the occurence of \(B\) wouldn’t have any effect on the probability of \(A\). In the bivariate case, the intersection is given by joint pmf \(f(x,y)\) and the event that would have an effect on how \(Y\) occurs is \(X\), and values of \(X\) can be got from its marginal pmf, \(f_X(x)\).

Example: Discrete Conditional Probability Function #1

A bivariate distribution has the following probability function:

$$ \begin{array}{c|c|c|c} {\quad \text X }& {0} & {1} & {2} \\ {\Huge \diagdown } & & & \\ {\text Y \quad} & & & \\ \hline {1} & {0.1} & {0.1} & {0.1} \\ \hline {2} & {0.1} & {0.1} & {0.2} \\ \hline {3} & {0.2} & {0.1} & {0.1} \end{array} $$

  1. Determine the marginal distribution of \(X\).
  2. Determine the marginal distribution of \(Y\).
  3. Evaluate the conditional distribution of \(Y|X=2\).

Solution

  1. The marginal distribution of \(X\) can be found by summing the columns in the table so that:
    $$\begin{align} P(X=0)&=0.1+0.1+0.2=0.4\\ P(X=1)&=0.1+0.1+0.1=0.3\\ P(X=2)&=0+0.2+0.1=0.3 \end{align}$$
    When represented using the table, the marginal distribution of \(X\) is:
    $$ \begin{array}{c|c|c|c} \text{X} & {0} & {1} & {2} \\ \hline {{P}({X}={x})} & {0.4} & {0.3} & {0.3} \end{array} $$
  2. The marginal distribution of \(Y\) can be found by summing the values in the rows of the table so that:
    $$ \begin{align} P(Y=1)&=0.1+0.1+0=0.2\\P(Y=2)&=0.1+0.1+0.2=0.4\\P(Y=3)&=0.2+0.1+0.1=0.4 \end{align} $$
    Therefore, the marginal distribution of \(Y\) is:
    $$ \begin{array}{c|c|c|c} \text{Y} & {1} & {2} & {3} \\ \hline {{P}({Y}={y})} & {0.2} & {0.4} & {0.4} \end{array} $$
  3. Using the definition of conditional probability:
    $$ \begin{align}P(Y=1|X=2)&=\cfrac { P(Y=1,X=2) }{ P(X=2) } =\cfrac { 0 }{ 0.3 } =0\\P(Y=2|X=2)&=\cfrac { P(Y=2,X=2) }{ P(X=2) } =\cfrac { 0.2 }{ 0.3 } =0.67\\ P(Y=3|X=2)&=\cfrac { P(Y=3,X=2) }{ P(X=2) } =\cfrac { 0.1 }{ 0.3 } =0.33\\ \end{align} $$
    The marginal distribution, is therefore:
    $$f(y|x=2)=\begin{cases}0, &Y=1\\ 0.67, &Y=2\\0.33, &Y=3\\ \end{cases}$$

Example: Discrete Conditional Probability Function #2

Suppose \(X\) and \(Y\) have the joint PMF given below,

$$ f(x,y) = \frac{5x+3y}{81}, \quad x=1,2, \quad y=1,2,3 $$

Find:

  1. \(h(y|x)\); and
  2. \(g(x|y)\).

Solution

  1. We know that:
    $$h(y|x)=\frac{f(x,y)}{f_X(x)}$$
    \(f_X(x)\) (the marginal pmf of X) is given by:
    $$f_X(x)=\sum_{y}{f(x,y)=P(X=x)}, x\epsilon S_x$$
    Therefore,
    $$\begin{align}f_X(x)&=\sum_{y=1}^{3}{\frac{5x+3y}{81}}\\ &==\frac{5x+3\left(1\right)}{81}+\frac{5x+3\left(2\right)}{81}+\frac{5x+3\left(32\right)}{81}\\ &=\frac{15x+18}{81}\end{align}$$
    $$\therefore f_X (x)=\frac{15x+18}{81},\ x=1,2$$
    Thus, the conditional pmf of \(Y\), given that \(X=x\), is equal to:
    $$h\left(y\middle| x\right)=\frac{\frac{5x+3y}{81}}{\frac{15x+18}{81}}=\frac{5x+3y}{15x+18},\  y=1,\ 2,\ 3,\ x=1, 2$$
  2. We know that \(f_Y(y)\) (marginal pmf of Y) is given by:
    $$\begin{align}f_Y\left(y\right)&=\sum_{x}{f\left(x,y\right)=P\left(Y=y\right),\ y\epsilon S_y}\\ &=\sum_{x=1}^{2}\frac{5x+3y}{81}\\ &=\frac{5\left(1\right)+3y}{81}+\frac{5\left(2\right)+3y}{81}\\ \therefore f_Y\left(y\right) &=\frac{6y+15}{81}\ y=1,2,3  \end{align}$$
    Then we know that:
    $$g\left(x\middle| y\right)=\frac{f\left(x,y\right)}{f_Y\left(y\right)}$$
    Thus,
    $$g\left(x\middle| y\right)=\frac{\frac{5x+3y}{81}}{\frac{15x+18}{81}}=\frac{5x+3y}{15x+18} \  \ x=1,2\ y=1,2,3 $$

Now that we know the conditional distributions, we can comfortably find the conditional probabilities. For example:

$$P\left(Y=1\middle| X=2\right)=h\left(1\middle|2\right)=\frac{13}{48}$$

If we find all the probabilities for these conditional probability functions we would see that they behave as the joint probability mass functions seen in the last chapter. Let’s keep \(X=2\) fixed and check this:

$$ P(Y=2|X=2)= h(2|2)=\frac{16}{48} \text{ and } P(Y=3|X=2)= h(3|2)=\frac{19}{48} $$

Then, if we sum these values keeping X constant, we find that:

$$ P(Y|X=2) = h(1|2) + h(2|2) + h(3|2) = \frac{13}{48} + \frac{16}{48} + \frac{19}{48} = 1 $$

Note that the same condition occurs for the \(g(x|y)\) conditional function.

Thus, \(h(y|x)\) and \(g(x|y)\) both satisfy the conditions of a probability mass function, and we can do the same operations we did on a joint pmf, such as:

$$ P(a < Y < b| X = x) = \sum_{\{y:a < y < b\}} h(y|x) $$

and conditional expectations such as

$$ E[u(Y)|X = x] = \sum_{y}u(y)h(y|x) $$

in a manner similar to those associated with unconditional probabilities and expectations.

Two special conditional moments are the conditional meanof \(Y\), given that \(X = x\), defined by:

$$ \mu_{Y|x}=E(Y|x)=\sum_{y}yh(y|x), $$

and the conditional variance of \(Y\), given that \(X = x\), defined by:

$$ \sigma_{Y|x}^2 = E\{[Y – E(Y|x)]^2|x\} = \sum_{y}[y-E(Y|x)]^2h(y|x), $$

We can compute this as:

$$ \sigma_{Y|x}^2 = E(Y^2|x) – [E(Y|x)]^2 $$

We would use the same logic to find the conditional mean \(\mu_{X|y}\) and the conditional variance \(\sigma_{X|y}^2\).

Using the values we found in example 2, let’s compute the conditional mean and variance:

\begin{align} \mu_{Y|2} & = E(Y|X=2) = \sum_{y=1}^{3}yh(y|2)\\ & = \sum_{y=1}^{3}y\left(\frac{10+3y}{48}\right) = 1 \left(\frac{13}{48}\right) + 2 \left(\frac{16}{48}\right) + 3 \left(\frac{19}{48}\right)\\ & = \frac{17}{8} \end{align}

and

\begin{align} \sigma_{Y|2}^2 & = E\left[\left(Y – \frac{17}{8}\right)^2\bigg|X=2\right] = \sum_{y=1}^{3}\left(y-\frac{17}{8}\right)^2\left(\frac{10+3y}{48}\right)\\ & = \left(1-\frac{17}{8}\right)^2\frac{13}{48}+\left(2-\frac{17}{8}\right)^2\frac{16}{48}+\left(3-\frac{17}{8}\right)^2\frac{19}{48}\\ & = \left(\frac{81}{64}\right)\frac{13}{48}+\left(\frac{1}{64}\right)\frac{16}{48}+\left(\frac{49}{64}\right)\frac{19}{48} \approx 0.651 \end{align}

Continuous Conditional Functions

If \(X\) and \(Y\) have a joint probability density function \( \text{f}(\text{x},\text{y}) \), then the conditional probability density function of \(X\) given that \( \text{Y}=\text{y}\) is defined, for all values of \(y\) such that \( \text{f}_\text{y}(\text{y})>0 \), by:

$$ \text{g}\left(\text{x}|\text{y}\right)=\frac{\text{f}\left(\text{x},\text{y}\right)}{\text{f}_{\text{Y}}(\text{y})} $$

Similarly, the conditional probability density function of \(Y\) given \(X\), \( \text{h}(\text{y}|\text{x}) \), is:

$$ \text{h}\left(\text{y}|\text{x}\right)=\frac{\text{f}\left(\text{x},\text{y}\right)}{\text{f}_\text{X}\left(\text{x}\right)} $$

Example: Continuous Conditional Functions

The random variables \(X\) and \(Y\) have a joint density function:

$$ \text{f}\left(\text{x},\text{y}\right)=\frac{1}{20}\left(2\text{x}+5\text{y}\right) \ \ \ 0 < \text{x} < 2,\ 0<\text{y} $$

Find the conditional density function of \(X\) given \(\text{Y}=\text{y} \).

Solution

We know that:

$$ \text{g}\left(\text{x}|\text{y}\right)=\frac{\text{f}\left(\text{x},\text{y}\right)}{\text{f}_\text{Y}(\text{y})} $$

Now,

$$ \begin{align*} \text{f}_\text{Y}\left(\text{y}\right)&=\int_{-\infty}^{\infty}{\text{f}\left(\text{x},\text{y}\right)\text{dx},\ \ \ \ \ \ \ \text{y}\epsilon \text{S}_\text{y}}, \\ &=\int_{\text{x}=0}^{2}{\frac{1}{20}(2\text{x}+5\text{y})\text{dx}} \\ &=\frac{1}{20}\left[\frac{2}{2}\text{x}^2+5\text{xy}\right]_{\text{x}=0}^2 \\ &=\frac{1}{20}(\left[2^2+5\text{y}\times2\right]-{[2}^0+5\text{y}\times0]) \\ &=\cfrac {(4+10\text{y})}{20} \\ \therefore \text{f}_\text{Y}\left(\text{y}\right) & =\frac{4+10\text{y}}{20}=\frac{2+5\text{y}}{10} \end{align*} $$

So,

$$ \begin{align*} \text{g}\left(\text{x}|\text{y}\right)&=\frac{\frac{1}{20}\left(2\text{x}+5\text{y}\right)}{\frac{1}{20}\left(4+10\text{y}\right)}=\frac{2\text{x}+5\text{y}}{2(2+5\text{y})} \\ &=\frac{2\text{x}+5\text{y}}{2(2\text{x}+5\text{y})},\ \ 0<\text{x} < 2 \end{align*} $$

The conditional pdf of \(Y\) given that \( \text{X}=\text{x} \) is:

$$ \text{h}\left(\text{y}|\text{x}\right)=\frac{\text{f}_{\text{XY}}\left(\text{x},\text{y}\right)}{\text{f}_\text{X}\left(\text{x}\right)},\ \ \ \ \ \text{ provided that } \text{f}_\text{X}\left(\text{x}\right)>0 $$

Learning Outcome

Topic 3.b: Multivariate Random Variables – Determine conditional and marginal probability functions, probability density functions, and cumulative distribution functions.

Shop CFA® Exam Prep

Offered by AnalystPrep

Featured Shop FRM® Exam Prep Learn with Us

    Subscribe to our newsletter and keep up with the latest and greatest tips for success
    Shop Actuarial Exams Prep Shop GMAT® Exam Prep


    Daniel Glyn
    Daniel Glyn
    2021-03-24
    I have finished my FRM1 thanks to AnalystPrep. And now using AnalystPrep for my FRM2 preparation. Professor Forjan is brilliant. He gives such good explanations and analogies. And more than anything makes learning fun. A big thank you to Analystprep and Professor Forjan. 5 stars all the way!
    michael walshe
    michael walshe
    2021-03-18
    Professor James' videos are excellent for understanding the underlying theories behind financial engineering / financial analysis. The AnalystPrep videos were better than any of the others that I searched through on YouTube for providing a clear explanation of some concepts, such as Portfolio theory, CAPM, and Arbitrage Pricing theory. Watching these cleared up many of the unclarities I had in my head. Highly recommended.
    Nyka Smith
    Nyka Smith
    2021-02-18
    Every concept is very well explained by Nilay Arun. kudos to you man!
    Badr Moubile
    Badr Moubile
    2021-02-13
    Very helpfull!
    Agustin Olcese
    Agustin Olcese
    2021-01-27
    Excellent explantions, very clear!
    Jaak Jay
    Jaak Jay
    2021-01-14
    Awesome content, kudos to Prof.James Frojan
    sindhushree reddy
    sindhushree reddy
    2021-01-07
    Crisp and short ppt of Frm chapters and great explanation with examples.