Bayesian Analysis

After completing this reading you should be able to:

• Describe Bayes’ theorem and apply this theorem in the calculation of conditional probabilities.
• Compare the Bayesian approach to the frequentist approach.
• Apply Bayes’ theorem to scenarios with more than two possible outcomes and calculate posterior probabilities.

Overview

The Bayes’ theorem is a fundamental of the Bayesian analysis. Assuming that we have two random variables \(A\) and \(B\), then according to Bayes’ theorem:

$$ P\left( A|B \right) =\frac { P\left( B|A \right) \times P\left[ A \right] }{ P\left[ B \right] } $$

Applying Bayes’ Theorem

Supposing that we are issued with two bonds \(A\) and \(B\). Each bond has a default probability of 10% over the year that follows. We are also told that there is a 6% chance that both the bonds will default, 86% chance that none of them defaults, and 4% chance that either of the bond defaults. All of this information can be summarized in a probability matrix.

Often, there is a high correlation between bond defaults. This can be attributed to the sensitivity displayed by bond issuers when dealing with broad economic bonds. The 6% chances of both the bonds defaulting are higher than the 1% chances of default had the default events been independent.

The features of the probability matrix can also be expressed in terms of conditional probabilities. For example, the likelihood that bond \(A\) will default given that \(B\) has defaulted is computed as:

$$ P\left( A|B \right) =\frac { P\left[ A\cap B \right] }{ P\left[ B \right] } =\frac { 6\% }{ 10\% } =60\% $$

This means that in 60% of the scenarios in which bond \(B\) will default, bond \(A\) will also default. The above equation is often written as:

$$ P\left[ A\cap B \right] =P\left( A|B \right) \times P\left[ B \right] \quad \quad \quad \quad I $$

Also:

$$ P\left[ A\cap B \right] =P\left( B|A \right) \times P\left[ A \right] \quad \quad \quad \quad II $$

Both the right-hand sides of equations \(I\), and \(II\) are combined and rearranged to give the Bayes’ theorem:

$$ P\left( A|B \right) =\frac { P\left( B|A \right) \times P\left[ A \right] }{ P\left[ B \right] } $$

When presented with new data, the Bayes’ theorem can be applied to update beliefs about the globe. To understand how the theorem can provide a framework for how exactly the new beliefs should be, consider the following scenario:

Based on an examination of historical data, it’s been determined that all fund managers at a certain Fund fall into one of two groups: Stars and Non-Stars. Stars are the best managers. The probability that a Star will beat the market in any given year is 75%. Other managers are just as likely to beat the market as they are to underperform it [i.e., Non-Stars have 50/50 odds of beating the market. For both types of managers, the probability of beating the market is independent from one year to the next. Stars are rare. Of a given pool of managers, only 16% turn out to be Stars.

A new manager was added to the portfolio of funds three years ago. Since then, the new manager has beaten the market every year. What was the probability that the manager was a star when the manager was first added to the portfolio? What is the probability that this manager is a star now? What’s the probability that the manager will beat the market next year, given that he has beaten it in the past three years?

Solution:

We first summarize the data by introducing some notations as follows: The chances that a manager will beat the market on the condition that he is a star is:

$$ P\left( B|S \right) =0.75=\frac { 3 }{ 4 } $$

The chances of a non-star manager beating the market are:

$$ P\left( B|\bar { S } \right) =0.5=\frac { 1 }{ 2 } $$

The chances of the new manager being a star during the particular time he was added to the analyst’s portfolio are exactly the chances that any manager will be made a star, which is unconditional:

$$ P\left[ S \right] =0.16=\frac { 4 }{ 25 } $$

To evaluate the likelihood of him being a star at present, we compute the likelihood of him being a star given that he has beaten the market three for three consecutive years, \(P\left( S|3B \right) \), using the Bayes’ theorem:

$$ P\left( S|3B \right) =\frac { P\left( 3B|S \right) \times P\left[ S \right] }{ P\left[ 3B \right] } $$

$$ P\left( 3B|S \right) ={ \left( \frac { 3 }{ 4 } \right) }^{ 3 }=\frac { 27 }{ 64 } $$

The unconditional chances that the manager will beat the market for three years is the denominator.

$$ P\left[ 3B \right] =P\left( 3B|S \right) \times P\left[ S \right] +P\left( 3B|\bar { S } \right) \times P\left[ \bar { S } \right] $$

$$ P\left[ 3B \right] ={ \left( \frac { 3 }{ 4 } \right) }^{ 3 }\times \frac { 4 }{ 25 } +{ \left( \frac { 1 }{ 2 } \right) }^{ 3 }\frac { 21 }{ 25 } =\frac { 69 }{ 400 } $$

Therefore:

$$ P\left( S|3B \right) =\frac { \left( \frac { 27 }{ 64 } \right) \left( \frac { 4 }{ 25 } \right) }{ \left( \frac { 69 }{ 400 } \right) } =\frac { 9 }{ 23 } =39\% $$

Therefore, there are 39% chances the manager will be a star after beating the market for three consecutive years, which happens to be our new belief and is a significant improvement to our old belief which was 16%.

Finally, we compute the chances of the manager beating the market in the following year. This happens to be the summation of the chances of a star beating the market and the chances of a non-star beating the market, weighted by the new belief:

$$ P\left[ B \right] =P\left( B|S \right) \times P\left[ S \right] +P\left( B|\bar { S } \right) \times P\left[ \bar { S } \right] $$

$$ P\left[ B \right] =\frac { 3 }{ 4 } \times \frac { 9 }{ 23 } +\frac { 1 }{ 2 } \times \frac { 14 }{ 23 } =60\%=\frac { 3 }{ 5 } $$

We also have that:

$$ P\left( S|3B \right) =\frac { P\left( 3B|S \right) \times P\left[ S \right] }{ P\left[ 3B \right] } $$

The L.H.S of the formula is posterior. The first item on the numerator is the likelihood, and the second part is the prior.

Bayes versus Frequentists

In the frequentists approach, we take three out of three results that are positive and make a conclusion that there are 100% chances of having a positive outcome in the year that follows. The observed frequency of the positive outcomes is the basis of this conclusion.

A similar scenario is witnessed in the Bayesian approach where the number of positive outcomes is counted. However, they have different conclusions where, as opposed to the Frequentists, a prior belief about the probability is the starting point of the Bayesian approach.

According to those who support the Bayesian Approach, the frequentist approach is absurd when used in small sets of data. The opponents of the Bayesian approach claim that Bayesian priors are too arbitrarily chosen, and are either subjective or based on analysis by the frequentists.

However, for most practitioners, it depends on the circumstances for them to decide which approach to apply. You will find them applying the Bayesian analysis in case the dataset is too little or when they have an extremely low signal-to-noise ratio. Examples of scenarios with little data that is noisy are stress testing and performance analysis.

With a huge dataset, the Bayesian and Frequentist approaches both yield similar conclusions, but the frequentists results are easier to compute.

Many-State Problems

The Bayesian analysis can be extended to any number of possibilities. For instance, assume that instead of the stars and non-star managers, we are given underperformers who can beat the market only a quarter of the time, the in-line performers who can beat the market only half the times, and the outperformers who beat the market three-quarters of the time. The original belief is that a particular manager has greater chances of being an inline performer as compared to being an outperformer or an underperformer.

There are 60% chances that a manager will be an in-line performer and 20% chances that he will be an underperformer, and 20% chances that he will be an outperformer. This implies that:

$$ P\left[ p=0.25 \right] =0.2 $$

$$ P\left[ p=0.5 \right] =0.6 $$

$$ P\left[ p=0.75 \right] =0.2 $$

We first wish to determine our updated beliefs in the event that the manager beats the market two years consecutively.To start off, we compute the likelihoods for each management category, which is the chances of beating the market in two consecutive years:

$$ P\left( 2B|p=0.25 \right) ={ \left( \frac { 1 }{ 4 } \right) }^{ 2 }=\frac { 1 }{ 16 } $$

$$ P\left( 2B|p=0.50 \right) ={ \left( \frac { 1 }{ 2 } \right) }^{ 2 }=\frac { 1 }{ 4 } =\frac { 4 }{ 16 } $$

$$ P\left( 2B|p=0.75 \right) ={ \left( \frac { 3 }{ 4 } \right) }^{ 2 }=\frac { 9 }{ 16 } $$

Now, given our prior beliefs about \(p\), we have the following unconditional chances of the manager beating the market for two consecutive years:

$$ P\left[ 2B \right] =0.2\times \frac { 1 }{ 16 } +60\%\times \frac { 4 }{ 16 } +0.2\times \frac { 9 }{ 16 } =\frac { 11 }{ 40 } $$

The belief that the manager is an underperformer can be computed by combining all this and applying the Bayesian theorem:

$$ P\left( p=0.25|2B \right) =\frac { P\left( 2B|p=0.25 \right) \times P\left[ p=0.25 \right] }{ P\left[ 2B \right] } $$

$$ =\frac { \frac { 1 }{ 16 } \times \frac { 2 }{ 10 } }{ \frac { 11 }{ 40 } } =4.55\% $$

To compute the posterior chances of the manager being an in-line performer we have that:

$$ P\left( p=0.50|2B \right) =\frac { P\left( 2B|p=0.5 \right) \times P\left[ p=0.50 \right] }{ P\left[ 2B \right] } $$

$$ =\frac { \frac { 4 }{ 16 } \times \frac { 6 }{ 10 } }{ \frac { 11 }{ 40 } } =54.55\% $$

To compute the posterior chances of the manager being an outperformer we have that:

$$ P\left( p=0.75|2B \right) =\frac { P\left( 2B|p=0.75 \right) \times P\left[ p=0.75 \right] }{ P\left[ 2B \right] } $$

$$ =\frac { \frac { 9 }{ 16 } \times \frac { 2 }{ 10 } }{ \frac { 11 }{ 40 } } =40.91\% $$

You will notice that all the probabilities sum up to 100%.

It is also crucial to point out that, the posterior probability was computed as follows for each type of manager:

$$ P\left( p=X|2B \right) =\frac { P\left( 2B|p=X \right) \times P\left[ p=X \right] }{ P\left[ 2B \right] } $$

We can observe that the R.H.S denominator is the same in each case, \( P\left[ 2B \right]\). This equation can also be rewritten as follows using a constant \(c\):

$$ P\left( p=X|2B \right) =c\times P\left( 2B|p=X \right) \times P\left[ p=X \right] $$

Furthermore, it is also an established fact that all the posterior probabilities must sum up to one. That is:

$$ \sum _{ i=1 }^{ 3 }{ c\times P\left( 2B|p={ X }_{ i } \right) \times P\left[ p={ X }_{ i } \right] } $$

$$ =c\sum _{ i=1 }^{ 3 }{ P\left( 2B|p={ X }_{ i } \right) \times P\left[ p={ X }_{ i } \right] } =1 $$

With this knowledge, solving problems that are seemingly intractable becomes quite simple, with continuous distributions.

Question

Suppose you are an equity analyst for XYZ investment bank. You use historical data to categorize the managers as excellent or average. Excellent managers outperform the market 70% of the time and average managers outperform the market only 40% of the time. Furthermore, 20% of all fund managers are excellent managers and 80% are simply being average. The probability of a manager outperforming the market in any given year is independent of their performance in any other year.

A new fund manager started three years ago and outperformed the market all three years. What’s the probability that the manager is excellent?

1. 29.53%
2. 12.56%
3. 57.26%
4. 30.21%

The correct answer is C.

The best way to visualize this problem is to start off with a matrix:

Let E be the event of an excellent manager, and A represent the event of an average manager.
P(E) = 0.2 and P(A) = 0.8

Further, let O be the event of outperforming the market.

We know that:

P(O|E) = 0.7 and P(O|A) = 0.4

We want P(E|O):

$$ P\left( E|O \right) =\frac {P\left( O|E \right) \times P(E)}{ P\left( O|E \right) \times P(E) + P\left( O|E \right) \times  P(A) } $$

$$ P\left( E|O \right) =\frac {\left( 0.7^{ 3 }\right) \times 0.2}{ \left( 0.7^{ 3 }\right) \times 0.2 + \left( 0.4^{ 3 }\right) \times 0.8 } = 57.26 \% $$

Note: The power of three to indicate three consecutive years