Common Univariate Random Variables

After completing this reading, you should be able to:

• Distinguish the key properties among the following distributions: uniform distribution, Bernoulli distribution, Binomial distribution, Poisson distribution, normal distribution, lognormal distribution, Chi-squared distribution, Student’s t, and F-distributions, and identify common occurrences of each distribution.
• Describe a mixture distribution and explain the creation and characteristics of mixture distributions.

Parametric Distributions

There are two types of distributions, namely parametric and non-parametric distributions. Functions mathematically describe parametric distributions. On the other hand, one cannot use a mathematical function to describe a non-parametric distribution. Examples of parametric distributions are uniform and normal distributions.

Discrete Random Variables

Bernoulli Distribution

A Bernoulli distribution is a discrete random variable that takes on values of 0 and 1. This distribution is suitable for scenarios with binary outcomes, such as corporate defaults. Most of the time, 1 is always labeled “success” and 0 a “failure.”

The Bernoulli distribution has a parameter p which is the probability of success, i.e., the probability that X=1, then:

$$P\left[ X=1 \right] =p\quad and\quad P\left[ X=0 \right] =1-p$$

The probability mass function of the Bernoulli distribution stated as $$X \sim Bernoulli \left(p \right)$$  is given by:

$$f_X \left(x \right) =p^x \left(1-p \right)^{1-x}$$

Therefore, the mean and variance of the distribution are computed as:

The PMF confirms that:
$$P [X=1]=p \text{and} P[X=0]=1-p$$

The CDF of a Bernoulli distribution is a step function given by:

$$F_X (x)=\begin{cases} &0, y<0\\&1-p, 0 ≤ y<1\\&1, y≥1\\\end{cases}$$

Therefore, the mean and variance of the distribution are computed as:

$$E \left(X \right)=p×1+(1-p)×0=p$$

$$V(X)=E(X^2 )-[E(X)]^2=[p×1^2+(1-p)×0^2 ]-p^2=p(1-p)$$

Example: Bernoulli Distribution

What is the ratio of the mean to variance for X~Bernoulli(0.75)?

Solution

We know that for Bernoulli Distribution,

$$E(X)=p$$

and

$$V(X)=p(1-p)$$

So,

$$\frac{E(X)}{V(X)}=\frac{p}{p(1-p)}=\frac{1}{0.25}=4$$

Thus, E(X): V(X)=4:1

Binomial Distribution

A binomial distribution is a collection of Bernoulli random variables. A binomial random variable quantifies the total number of success from a n independent Bernoulli random variable, with the probability of success be p and of course, the failure being 1-p. Consider the following example:

Suppose we are given two totally independent bonds, with a default likelihood of ten percent. Then we have the following possibilities:

• both of the bonds do not default
• both of them defaults, or
• only one of them defaults.

Let $$X$$ represent the number of defaults:

$$P\left[ X=0 \right] ={ \left( 1-10\% \right) }^{ 2 }=81\%$$

$$P\left[ X=1 \right] =2\times 10\%\times \left( 1-10\% \right) =18\%$$

$$P\left[ X=2 \right] ={ 10\% }^{ 2 }=1\%$$

If we possess three independent bonds having a 10% default probability then:

$$P\left[ X=0 \right] ={ \left( 1-10\% \right) }^{ 3 }=72.9\%$$

$$P\left[ X=1 \right] ={ 3\times 10\%\times \left( 1-10\% \right) }^{ 2 }=24.3\%$$

$$P\left[ X=2 \right] ={ 3\times { 10\% }^{ 2 }\times \left( 1-10\% \right) }=2.7\%$$

$$P\left[ X=3 \right] ={ { 10\% }^{ 3 } }=0.1\%$$

Suppose now that we have $$n$$ bonds. The following combination represents the number of ways in which $$k$$ of the $$n$$ bonds can default:

$$\left( \begin{matrix} n \\ x \end{matrix} \right) =\frac { n! }{ x!\left( n-x \right) ! } \quad \quad \dots \dots \dots \dots equation\quad I$$

If $$p$$ is the likelihood that one bond will default, then the chances that any particular $$k$$ bonds will default is given by:

$${ p }^{ x }{ \left( 1-p \right) }^{ n-x }\quad \quad \dots \dots \dots \dots \dots equation\quad II$$

Combining equation $$I$$ and $$II$$, we can determine the likelihood of $$k$$ bonds defaulting as follows:

$$P\left[ X=x \right] =\left( \begin{matrix} n \\ x \end{matrix} \right) { p }^{ x }{ \left( 1-p \right) }^{ n-x } = \left(\begin{matrix} n \\ x\end{matrix} \right) p^x (1-p)^{n-x} \text{for} \text{x} = 0,1,2,\dots n$$

This is the PDF for the binomial distribution.

Therefore, binomial distribution has two parameters: n and p and usually stated as $$X~B(n,p)$$

The CDF of a binomial distribution is given by:

$$\sum_{i=1}^{|x|} { \left(\begin{matrix} n \\ i \end{matrix} \right) p^i (1-p)^{n-i}}$$

Where |x| implies a random variable less than or equal to x.

The mean and variance of the binomial distribution can be evaluated to using moments. The mean and variance are given by:

$$E(E)=np$$

And

$$V(X)=np(1-p)$$

The binomial can be approximated using a normal distribution (as will be seen later) if $$np≥10$$ and $$n(1-p)≥10$$

Example: Binomial Distribution

Consider a Binomial distribution X~B(4,0.6). Calculate P(X≥ 3).

Solution

We know that for binomial distribution:

$$P\left[ X=x \right]= \left(\begin{matrix} n \\ x\end{matrix} \right) p^x (1-p)^{n-x}$$

In this case, $$n=4$$ and $$p=0.6$$

$$⇒P(X≥3)=P(X=3)+P(X=4)= \left(\begin{matrix} 4 \\ 3 \end{matrix} \right) p^3 (1-p)^{4-3}+ \left(\begin{matrix} 4 \\ 4 \end{matrix} \right) p^4 (1-p)^{4-4}$$

$$= \left(\begin{matrix} 4 \\ 3 \end{matrix} \right) 0.6^3 (1-0.6)^{4-3}+ \left(\begin{matrix} 4 \\ 4 \end{matrix} \right) 0.6^4 (1-0.6)^{4-4}$$

$$=0.3456+0.05184=0.39744$$

Poisson Distribution

Events are said to follow a Poisson process if they happen at a constant rate over time, and the likelihood that one event will take place is independent of all the other events for instance number of defaults that occur in each month

Suppose that X is a Poisson random variable, stated as X~Poisson(λ) then the PMF is given by:

$$P\left[ X=x \right] =\frac { { \lambda }^{ x } e ^{ -\lambda } }{ x! }$$

The CDF of a Poisson distribution is given by:

$$\sum_{i=1}^{|x|} {\frac{{\lambda}^i}{i!}}$$

The Poisson parameter λ (lambda), termed as the hazard rate, represents the mean number of events in an interval. Therefore, the mean and variance of the Poisson distribution are given by:

$$E(X)=λ$$

And

$$V(X)=λ$$

Example: Poisson Distribution

A fixed income portfolio is made of a huge number of independent bonds. The average number of bonds defaulting every month is 10. What is the probability that exactly 5 defaults in one month?

Solution

For Poisson distribution:

$$P(X=x)=\frac { { \lambda }^{ x } e ^{ -\lambda } }{ x! }$$

For this question, we have that: $$λ=10$$ and we need:

$$P(X=5)=\frac { { 10 }^{ 5 } e ^{ -10 } }{ 5! }=0.03783$$

The notable feature of Poisson distribution is that it is infinitely divisible. That is, if $$X_1~\text{Poisson} (λ_1)$$ and $$X_2~\text{Poisson}(λ_2)$$ and that $$Y=X_1+X_2$$  then,

$$Y \sim \text{Poisson}(λ_1+λ_2)$$

Therefore, Poisson distribution is suitable for time series data since summing the number of events in the sampling interval does not distort the distribution.

Continuous Random Variables

Uniform Distribution

A uniform distribution is a continuous distribution, which takes any value within the range [a,b], which equally likely to occur.

The PDF of a uniform distribution is given by:

$$f_X \left(x \right)=\frac{1}{b-a}$$

Note that the PDF  of a uniform random variable does not depend on x since all values are equally likely. The CDF of the uniform distribution

The CDF of  the uniform distribution is:

$$F_X(x)=\begin{cases} &0, x<a \\ &\frac{x-a}{b-a}, a≤x≤b \\ &1, x≥b \end{cases}$$

When a=0 and b=1 is called the standard uniform distribution denoted as, $$U_1$$ . From this distribution, we can construct ay uniform distribution, $$U_2$$ and $$U_1$$ using the formula:

$$U_2=a+ \left(b-a \right) U_1$$

Where a and b are limits of $$U_2$$

The uniform distribution is denoted by $$X \sim U(a,b)$$ and the mean and variance are given by:

$$E(X)= \frac{a+b}{2}$$

And the variance is given by:

$$V(X)=\frac{(b-a)^2}{12}$$

For instance, the variance of the standard uniform distribution $$U_1\sim N(0,1)$$ is given by:

$$E(X)=\frac{(0+1)}{2}=\frac{1}{2}$$

And

$$V(X)=\frac{(1-0)^2}{12}=\frac{1}{12}$$

Assume that we want to calculate the probability that X falls in the interval $$l<X<u$$ where l is lower limit and u is the upper limit. That is, we need $$P(l<X<u)$$   Given the $$X \sim U(a,b)$$. To compute this, we use the formula:

$$P(l<X<u)=\frac{min(u,b)-max(l,a)}{b-a}$$

Intuitively, if  $$l≥a$$ and $$u≤b$$, the formula above simplifies into:

$$\frac{u-l}{b-a}$$

Given the uniform distribution $$X~U(-5,10)$$, calculate the mean, variance, and $$P(-3<X<6)$$.

Solution

For uniform distribution,

$$E(X)= \frac{a+b}{2}=\frac{-5+10}{2}=2.5$$

And

$$V(X)=\frac{(10–5)^2}{12}=\frac{225}{12}=18.75$$

For $$P(-3<X<6)$$, using the formula:

$$P(l<X<u)=\frac{min(u,b)-max(l,a)}{b-a}$$

$$P(-3<X<6)=\frac{min(6,10)-max(-3,-5)}{10–5}=\frac{6–3}{10–5}=\frac{9}{15}=0.60$$

Alternatively, you can think the probability as the area under the curve. Note that the height of the uniform distribution is $$frac{1}{b-a}$$ and the length $$u-l$$.

That is:

$$\frac{1}{b-a} \times (u-l)= \frac{1}{10–5} \times (6–3)=\frac{9}{15}=0.60$$

Normal Distribution

Also called the Gaussian distribution, the normal distribution has a symmetrical PDF and the mean and median coincide with the highest point of the PDF. Furthermore, the normal distribution always has a skewness of 0 and a kurtosis of 3.

The following is the formula of a PDF that is normally distributed, for a given random variable $$X$$:

$$f\left( x \right) =\frac { 1 }{ \sigma \sqrt { 2\pi } } { e }^{ -\cfrac { 1 }{ 2 } { \left( \cfrac { x-\mu }{ \sigma } \right) }^{ 2 } }, -\infty <x< \infty$$

When a variable is normally distributed, it is often written as follows, for convenience:

$$X\sim N\left( \mu ,{ \sigma }^{ 2 } \right)$$

Where $$E(X)= μ$$ and $$V(X)= σ^2$$

We read this as $$X$$ is normally distributed, and has a mean, $$\mu$$,and a variance of $${ \sigma }^{ 2 }$$. Any linear combination of independent normal variables is also normal. To illustrate this, assume $$X$$ and $$Y$$ are two variables that are normally distributed. We also have constants $$a$$ and $$b$$. Then $$Z$$ will be normally distributed such that:

$$Z=aX+bY,\quad such\quad that\quad Z\sim N\left( a{ \mu }_{ X }+b{ \mu }_{ Y },{ a }^{ 2 }{ \sigma }_{ X }^{ 2 }+{ b }^{ 2 }{ \sigma }_{ Y }^{ 2 } \right)$$

For instance for $$a=b=1$$, then  $$Z=X+Y$$ and thus $$Z \sim N(μ_X+μ_Y,σ_X^2+σ_Y^2)$$

A standard normal distribution is a normal distribution whose mean is 0 and standard deviation is 1.It is denoted by N(0,1)

$$\emptyset =\frac { 1 }{ \sqrt { 2\pi } } { e }^{ -\frac { 1 }{ 2 } { x }^{ 2 } }$$

To determine a normal variable whose standard deviation is $$\sigma$$ and mean is $$\mu$$, we compute the product of the standard normal variable with $$\sigma$$ and then add the mean:

$$X=\mu +\sigma \emptyset \Rightarrow X\sim N\left( \mu ,{ \sigma }^{ 2 } \right)$$

Three standard normal variables $${ X }_{ 1 }$$, $${ X }_{ 2 }$$, and $${ X }_{ 3 }$$ are combined in the following way to construct two normal variables that are correlated:

$${ X }_{ A }=\sqrt { \rho } { X }_{ 1 }+\sqrt { 1-\rho } { X }_{ 2 }$$

$${ X }_{ B }=\sqrt { \rho } { X }_{ 1 }+\sqrt { 1-\rho } { X }_{ 3 }$$

Where $${ X }_{ A }$$ and $${ X }_{ B }$$ have a correlation of $$\rho$$, and are standard normal variables.

The z-value measures how many standard deviations the corresponding x value is above or below the mean. The z-value measures how many standard deviations the corresponding x value is above or below the mean. It is given by:

$$\Phi \left(z \right)=\frac{X-\mu}{\sigma} \sim N(0,1)$$

And

$$X \sim N(\mu {\sigma}^2)$$

Converting X normal random variables is termed as standardization. The values of z is usually tabulated.

For example,  consider the normal distribution X~N(1,2). We wish to calculate P(X>2).

Solution

For

$$P(X>2)=1-P(X≤2)=1-\frac{2-1}{\sqrt{2}}=0.2929\approx 0.29$$

We look up this value from the z-table.

$$\Phi \left(0.29\right) \approx 24.0 \%$$

 x-value z-value μ 0 μ+1σ 1 μ+2σ 2 μ+nσ n

Recall that the binomial random variable if $$np≥10$$ and $$n(1-p)≥10$$. If this is true, then the binomial distribution is normally distributed as:

$$X \sim N \left(np, n(1-p) \right)$$

Also, Poisson distribution is normally approximated is λ≥1000 so that: