Fundamentals of Probability

After completing this reading, you should be able to:

  • Describe an event and an event space.
  • Describe independent events and mutually exclusive events.
  • Explain the difference between independent events and conditionally independent events.
  • Calculate the probability of an event for a discrete probability function.
  • Define and calculate a conditional probability.
  • Distinguish between conditional and unconditional probabilities.
  • Explain and apply Bayes’ rule.

Probability is the foundation statistics, risk management, and econometrics. Probability quantifies the likelihood that some event will occur or occurs. For instance, the probability that there will be a defaulter in a prime mortgage.

Sample Space, Event Space, and Events

Sample Space (Ω)

A sample space is defined as a collection (set) containing all possible occurrences of the experiment. The outcomes are dependent on the problem in question. For example,  when modeling returns from a portfolio, the sample space is a set of real numbers. Additionally, if we want to models defaults in loan payment, the sample space could be stated as Ω={Default, No Default}.

Events (ω)

Events defined as the subsets of the sample space and denoted by ω (which is the small letter of Ω). An event is a set of outcomes (which may contain more than one element). Note that an event can be empty (∅). An event contains one outcome; it is termed an elementary event. For example, in two-dice rolling, we may wish to know the number of outcomes whose sum is odd. Then our event is denoted as: ω={(1,2), (1,4), (1,6)…(4,5), (5,6)}

Event Space (F)

Event space is a collection of all combinations of outcomes to which probabilities can be allocated. For example, consider the event spaces {A}, {B} and {C}. We may wish to calculate the probability of {A} occurring while {B} does not or probability of {B} occurring while {A} does not or the probability of both {A} and {B} or probability of neither {A} nor {B} occurring. These probabilities constitute a discrete probability space since it involves a finite number of outcomes.

Probability

As mentioned earlier, probability measure the likelihood of an event. Probabilities always lie between 0 and 1, inclusively.

According to frequentist interpretation, the probability is the number of occurrences of an event would occur if a set independent experiment is performed. The frequentist interpretation is just a conceptual explanation but in finance, we deal with a real scenario such as return from portfolios.

The subjective interpretation of probability is based on an individual’s beliefs about the likelihood of an event occurring, which may differ from one individual to another. For example, a risk manager might assume that the probability of interest rates falling is 70% while another risk manager may perform scenario analysis and believes the probability of interest rates falling is 48%.

Mathematically, a probability is defined over event spaces, where each event is assigned a number between 0 and 1. Recall that events is a combination of outcomes. The combination can be in form intersections and unions.

Independent and Mutually Exclusive Events

Mutually Exclusive Events

Two events, A and B, are said to be mutually exclusive if the occurrence of A rules out the occurrence of B, and vice versa. For example, a car cannot turn left and turn right at the same time.

Mutually Exclusive Events - ExampleIndependent events

Two events, A and B, are independent if the fact that A occurs does not affect the probability of B occurring. When two events are independent, this simply means that both events can happen at the same time. In other words, the probability of one event happening does not depend on whether the other event to occur. For example, we can define A as the likelihood that it rains on March 15 in New York and B as the probability that it rains in Frankfurt on March 15. In this instance, both events can happen simultaneously or not.

Another example would be defining event A as getting tails on the first coin toss and B on the second coin toss. The fact of landing on tails on the first toss will not affect the probability of getting tails on the second toss.

Intersection

The intersection of events, say A and B, is the set of outcomes occurring both in A and B. It is denoted as P(A∩B). Using the Venn diagram, this is represented as:

Intersection of EventsFor independent events,

$$P(A∩B)=P(A \text{ and } B)=P(A)×P(B)$$

Independence can be extended to n independent events: Let \(A_1,A_2,…,A_n\) be independent events then:

$$P(A_1∩A_2∩…∩ A_n  )=P(A_1)×P(A_2 )×…×P(A_n )$$

For mutually exclusive events,

$$P(A∩B)=P(A \text{ and } B)=0$$

since the occurrence of A rules out the occurrence of B. Remember that a car cannot turn left and turn right at the same time!

Union

The union of events, say A and B, is the set of outcomes occurring in at least one of the two sets – A or B. It is denoted as P(A∪B). Using the Venn diagram, this is represented as:

Union of EventsTo determine the likelihood of any two mutually exclusive events occurring, their individual probabilities will just be summed up. The following is the statistical notation:

$$ P\left( A\cup B \right) =P(A \text{ or } B)=P\left( A \right) +P\left( B \right) $$

Given two events A and B that are not mutually exclusive (independent events), the probability that at least one of the events will occur is given by:

$$P(A \cup B)=P(A \text{ or } B) = P(A)+P(B)-P(A \cap B)$$

Complement of a Set

Another important if the complement of a set. Denoted by Ac  (where A can be any other event) in the set of outcomes that are not in A. Consider the following Venn diagram:

Complement of a SetThis is the first axiom of Probability, and it implies that:

$$P(A ∪ A^c )=P(A)+P(A^c )=1$$

Conditional Probability

Until now, we’ve only looked at unconditional probabilities. An unconditional probability (also known as a marginal probability) is simply the probability that an event occurs, without taking into account any other preceding events. In other words, unconditional probabilities are not conditioned on the occurrence of any other events; they are ‘stand-alone’ events.

Conditional probability is the probability of one event occurring with some relationship to one or more other events. Our interest lies in the probability of an event ‘A’ given that another event ‘B ‘ has already occurred. This is what you should ask yourself:

“What is the probability of one event occurring if another event has already taken place?

We pronounce P(A | B) as “the probability of A given B.” and it is given by:

$$P(A│B)=\frac{P(A\cap B)}{P(B)}$$

The bar sandwiched between A and B simply indicates “given.”

 Example: Groups of investors

In a group of 100 investors,

  • 40 buy stocks,
  • 30 purchase bonds, and
  • 20 purchase stocks and bonds

If an investor chosen at random bought bonds, what is the probability they also bought stocks?

$$
\begin{array}{c|c|c}
\textbf{Event} & \textbf{Notation} & \textbf{Probability} \\
\hline
\text{Buys stocks} & \text{A} & \text{0.4(=40/100)} \\
\text{Buys bonds} & \text{B} & \text{0.3 (=30/100)} \\
\text{Buys stocks and bonds} & \text{A and B} & \text{0.2 (=20/100)} \\
\end{array}
$$

We want the probability of an investor buying stocks given that they have already bought bonds. This is P(A | B):

$$ P\left( A|B \right) =\frac { P\left( A\cap B \right) }{ P\left( B \right) } $$

$$ =\frac { 0.2 }{ 0.3 }  = 0.67$$

Note that we can also make the numerator the subject so that:

$$ P\left( A\cap B \right) =P\left( A|B \right) P\left( B \right) $$

For independent events, however,

$$ P\left( A|B \right) =\frac{P(A)P(B)}{P(B)}=P\left( A \right) $$

This is also true for \(P(B│A)=P(B)\).

Bayes’ Theorem

Bayes’ theorem describes the probability of an event, based on prior knowledge of conditions that might be related to the event. Assuming that we have two random variables A and B, then according to Bayes’ theorem:

$$ P\left( A|B \right) =\frac { P\left( B|A \right) \times P\left( A \right) }{ P\left( B \right) } $$

Applying Bayes’ Theorem

Supposing that we are issued with two bonds A and B. Each bond has a default probability of 10% over the year that follows. We are also told that there is a 6% chance that both the bonds will default, 86% chance that none of them defaults, and 4% chance that either of the bond defaults. All of this information can be summarized in a probability matrix.

Often, there is a high correlation between bond defaults. This can be attributed to the sensitivity displayed by bond issuers when dealing with broad economic bonds. The 6% chances of both the bonds defaulting are higher than the 1% chances of default had the default events been independent.

The features of the probability matrix can also be expressed in terms of conditional probabilities. For example, the likelihood that bond \(A\) will default given that \(B\) has defaulted is computed as:

$$ P\left( A|B \right) =\frac { P\left[ A\cap B \right] }{ P\left[ B \right] } =\frac { 6\% }{ 10\% } =60\% $$

This means that in 60% of the scenarios in which bond \(B\) will default, bond \(A\) will also default.

The above equation is often written as:

$$ P\left[ A\cap B \right] =P\left( A|B \right) \times P\left[ B \right] \quad \quad \quad \quad I $$

Also:

$$ P\left[ A\cap B \right] =P\left( B|A \right) \times P\left[ A \right] \quad \quad \quad \quad II $$

Both the right-hand sides of equations \(I\), and \(II\) are combined and rearranged to give the Bayes’ theorem:

$$P(B│A)×P[A]=P(A│B)×P(B)$$

$$ \Rightarrow P\left( A|B \right) =\frac { P\left( B|A \right) \times P\left[ A \right] }{ P\left[ B \right] } $$

When presented with new data, the Bayes’ theorem can be applied to update beliefs. To understand how the theorem can provide a framework for how exactly the new beliefs should be, consider the following scenario:

Example: Applying Baye’s theorem

Supposing that an analyst was able to group fund managers into two categories, star and non-star managers, after doing an evaluation of historical data. Given that the best managers are a star and there is a 75% likelihood that in a particular year, the market will be beaten by a star. On the other hand, there are equal chances that non-star managers will either beat the market or underperform it. Furthermore, there is a year to year independence in the probabilities that both types of managers will beat the market.

Only 16% of managers within a given cohort become stars. Three years have passed since a new manager who was able to beat the market every single year was added to the portfolio of the analyst. Determine the chances of the manager being a star when he was first added to the portfolio. What are the chances of him being a star at present? What are his chances of beating the market in the year that follows given that he has beaten it in the past three years?

Solution

We first summarize the data by introducing some notations as follows: The chances that a manager will beat the market on the condition that he is a star is:

$$ P\left( B|S \right) =0.75=\frac { 3 }{ 4 } $$

The chances of a non-star manager beating the market are:

$$ P\left( B|\bar { S } \right) =0.5=\frac { 1 }{ 2 } $$

The chances of the new manager being a star during the particular time he was added to the analyst’s portfolio are exactly the chances that any manager will be made a star, which is unconditional:

$$ P\left[ S \right] =0.16=\frac { 4 }{ 25 } $$

To evaluate the likelihood of him being a star at present, we compute the likelihood of him being a star given that he has beaten the market three for three consecutive years, \(P\left( S|3B \right) \), using the Bayes’ theorem:

$$ P\left( S|3B \right) =\frac { P\left( 3B|S \right) \times P\left[ S \right] }{ P\left[ 3B \right] } $$

$$ P\left( 3B|S \right) ={ \left( \frac { 3 }{ 4 } \right) }^{ 3 }=\frac { 27 }{ 64 } $$

The unconditional chances that the manager will beat the market for three years is the denominator.

$$ P\left[ 3B \right] =P\left( 3B|S \right) \times P\left[ S \right] +P\left( 3B|\bar { S } \right) \times P\left[ \bar { S } \right] $$

$$ P\left[ 3B \right] ={ \left( \frac { 3 }{ 4 } \right) }^{ 3 }\times \frac { 4 }{ 25 } +{ \left( \frac { 1 }{ 2 } \right) }^{ 3 }\frac { 21 }{ 25 } =\frac { 69 }{ 400 } $$

Therefore:

$$ P\left( S|3B \right) =\frac { \left( \frac { 27 }{ 64 } \right) \left( \frac { 4 }{ 25 } \right) }{ \left( \frac { 69 }{ 400 } \right) } =\frac { 9 }{ 23 } =39\% $$

Therefore, there are 39% chances the manager will be a star after beating the market for three consecutive years, which happens to be our new belief and is a significant improvement to our old belief which was 16%.

Finally, we compute the chances of the manager beating the market in the following year. This happens to be the summation of the chances of a star beating the market and the chances of a non-star beating the market, weighted by the new belief:

$$ P\left[ B \right] =P\left( B|S \right) \times P\left[ S \right] +P\left( B|\bar { S } \right) \times P\left[ \bar { S } \right] $$

$$ P\left[ B \right] =\frac { 3 }{ 4 } \times \frac { 9 }{ 23 } +\frac { 1 }{ 2 } \times \frac { 14 }{ 23 } =60\%=\frac { 3 }{ 5 } $$

We also have that:

$$ P\left( S|3B \right) =\frac { P\left( 3B|S \right) \times P\left[ S \right] }{ P\left[ 3B \right] } $$

The L.H.S of the formula is posterior. The first item on the numerator is the likelihood, and the second part is the prior. 

Question 1

The probability that the Eurozone economy will grow this year is 18%, and the probability that the European Central Bank (ECB) will loosen its monetary policy is 52%.

Assuming that the joint probability that the Eurozone economy will grow and the ECB will loosen its monetary policy is 45%, then what is the probability that either the Eurozone economy will grow or the ECB will loosen its the monetary policy?

A. 42.12%

B. 25%

C. 11%

D. 17%

The correct answer is B.

The addition rule of probability is used to solve this question:
P(E) = 0.18 (the probability that the Eurozone economy will grow is 18%)
p(M) = 0.52 (the probability that the ECB will loosen the monetary policy is 52%)
p(EM) = 0.45 (the joint probability that Eurozone economy will grow and the ECB will loosen its monetary policy is 45%)

The probability that either the Eurozone economy will grow or the central bank will loosen its the monetary policy: 
p(E or M) = p(E) + p(M) – p(EM) = 0.18 + 0.52 – 0.45 = 0.25

Question 2

A mathematician has given you the following conditional probabilities:

$$ \begin{array}{c|c} \text{p(O|T) = 0.62} & \text{Conditional probability of reaching the office if the train arrives on time} \\ \hline \text{p(O|T c) = 0.47} & \text{Conditional probability of reaching the office if the train does not arrive on time} \\ \hline \text{p(T) = 0.65} & \text{Unconditional probability of the train arriving on time} \\ \hline \text{p(O) = ?} & \text{Unconditional probability of reaching the office}  \\ \end{array} $$

What is the unconditional probability of reaching the office, p(O)?

A. 0.4325

B. 0.3265

C. 0.3856

D. 0.5244

The correct answer is A.

This question can be solved using the total probability rule.

If p(T) = 0.65 (Unconditional probability of train arriving at time is 0.65), then the unconditional probability of the train not arriving on time p(T c) = 1 – p(T) = 1 – 0.65 = 0.35.

Now, we can solve for p(O) = p(O|T) * p(T) + p(O|T c) * p(T c) = 0.62 * 0.65 + 0.47 * 0.35 = 0.5675

Question 3

Suppose you are an equity analyst for XYZ investment bank. You use historical data to categorize the managers as excellent or average. Excellent managers outperform the market 70% of the time and average managers outperform the market only 40% of the time. Furthermore, 20% of all fund managers are excellent managers and 80% are simply being average. The probability of a manager outperforming the market in any given year is independent of their performance in any other year.

A new fund manager started three years ago and outperformed the market all three years. What’s the probability that the manager is excellent?

A. 29.53%

B. 12.56%

C. 57.26%

D. 30.21%

The correct answer is C.

The best way to visualize this problem is to start off with a matrix:

$$ \begin{array}{l|c|c} \textbf{Kind of manager} & \textbf{Probability} & \textbf{Probability of beating market} \\
\hline \text{Excellent} & {0.2} & {0.7} \\
\hline \text{Average} & {0.8} & {0.4}  \\ \end{array} $$

Let E be the event of an excellent manager, and A represent the event of an average manager.
P(E) = 0.2 and P(A) = 0.8

Further, let O be the event of outperforming the market.

We know that:

P(O|E) = 0.7 and P(O|A) = 0.4

We want P(E|O):

$$ P\left( E|O \right) =\frac {P\left( O|E \right) \times P(E)}{ P\left( O|E \right) \times P(E) + P\left( O|E \right) \times  P(A) } $$

$$ P\left( E|O \right) =\frac {\left( 0.7^{ 3 }\right) \times 0.2}{ \left( 0.7^{ 3 }\right) \times 0.2 + \left( 0.4^{ 3 }\right) \times 0.8 } = 57.26 \% $$

Note: The power of three to indicate three consecutive years.


Leave a Comment

X