Bayes’ Formula

Bayes’ formula is used to calculate an updated or posterior probability given a set of prior probabilities for a given event. It’s a theorem named after Reverend T Bayes and it is widely used in Bayesian methods of statistical inference.

This is the logic used to come up with the formula:

Let E₁, E_2, E_3, …, E_n be a set of mutually exclusive and exhaustive events.

Using the conditional probability:

$$ P(E_i | A) =\cfrac {P(E_i A)}{P(A)} $$

And also the relationship:

$$ P(E_iA) = P(AE_i) = P(E_i)P(A | E_i) $$

And the total probability rule:

$$ P(A) = \sum {P(AE_j)} \quad \text { for all j} = 1, 2,…,n $$

We can finally substitute for P(E_iA) and P(A) in equation 1

This gives:

$$ P(E_{ i }|A)=\cfrac { P(E_{ i })P(A|E_{ i }) }{ \sum _{ j=1 }^{ n }{ P(E_{ i })P(A|E_{ i }) } } $$

This is the Bayes’ formula and it allows us to ‘turnaround’ conditional probabilities i.e., we can calculate P(E_i | A) if given information only about P(A | E_i).

Note:

1. P(E_j) are known as prior probabilities;
2. the event A is some event known to have occurred; and
3. P(E_i | A) is the posterior probability.

Example: Bayes’ Formula

A Civil Engineer wishes to investigate the punctuality of electric trains by considering the number of train journeys. In the sample, 50% of trains were destined for New York, 30% Vegas and 20% Washington DC. The probabilities of a train arriving late in New York, Vegas, and Washington DC are 40%, 35%, and 25% respectively. If the Engineer picks a train at random from this group, what is the probability that the destination of the train would be New York?

Solution:

We are looking for P(New York | Late).

Let’s define the following events:

First, N is the event “A train chosen at random whose destination is New York.”

Secondly, V is the event “A train chosen at random whose destination is Vegas.”

And W is the event “A train chosen at random whose destination is Washington DC.”

Finally, let L be the event “A randomly chosen train arrives late.”

$$ \begin{align*} P(N|L) & =\cfrac { P(N)P(L|N) }{ P(N)P(L|N)+P(V)P(L|V)+P(W)P(L|W) } \\ & =\cfrac { 0.5\ast 0.4 }{ 0.5\ast 0.4+0.3\ast 0.35+0.2\ast 0.25 } \\ & =\cfrac { 0.2 }{ 0.355 } \\ & =0.5634 \\ & =56.3\% \\ \end{align*} $$

We have computed P(N | L) given only P(L | N), hence the phrase ‘turnaround conditional probability’.

Question

A chartered analyst can choose any one of these three routes, A, B or C, to get to work. The probabilities that she arrives on time using routes A, B, and C are 50%, 52%, and 60% respectively. If she is equally likely to choose any one of the routes and arrive on time, calculate the probability that she chose route A.

A. 30.9%

B. 16.67%

C. 25%

Solution

The correct answer is A.

First, you should define the relevant events:

let: A be the event, “Chooses route A”;

B the event, “Chooses route B”; and

C the event, “Chooses route C”.

Lastly, define event T as, “Arrives to work on time”.

Now, what we have is P(T | A) =  i.e. the probability that the analyst arrives on time given that she chooses route A.

However, we want to find the turnaround probability P(A | T) i.e. the probability that the analyst chooses route A  given that she arrives on time.

Hence, applying Bayes’ formula:

$$ \begin{align*} P(A|T) & =\cfrac { P(A)P(T|A) }{ P(A)P(T|A)+P(B)P(T|B)+P(C)P(T|C) } \\ & =\cfrac { \frac {1}{3} \ast 0.5 }{ \frac {1}{3} \ast 0.5+\frac {1}{3} \ast 0.52+\frac {1}{3} \ast 0.6 } \\ & =\cfrac { 0.16667 }{ 0.54 } \\ & =0.30865 \\ & =30.9\% \\ \end{align*} $$

Tip: good understanding of Bayes’ theorem can only be preceded by a good understanding of the rules of probability. The focus should be on the calculations.

Reading 8 LOS 8n

Calculate and interpret an updated probability using Bayes’ formula.