Updating Probability Using Bayes’ Formula

Bayes’ formula is used to calculate an updated or posterior probability given a set of prior probabilities for a given event. It is a theorem named after the Reverend T Bayes and is used widely in Bayesian methods of statistical inference.

It is the logic used to come up with the formula:

Let \(E_1, E_2, E_3, …, E_n\) be a set of mutually exclusive and exhaustive events.

Using the conditional probability:

$$ P(E_i | A) =\cfrac {P(E_i A)}{P(A)} $$

And also the relationship:

$$ P(E_iA) = P(AE_i) = P(E_i)P(A | E_i) $$

And the total probability rule:

$$ P(A) = \sum {P(AE_j)} \quad \text { for all j} = 1, 2,…,n $$

We can finally substitute for \(P(E_iA)\) and \(P(A)\) in equation 1.

This gives:

$$ P(E_{ i }|A)=\cfrac { P(E_{ i })P(A|E_{ i }) }{ \sum _{ j=1 }^{ n }{ P(E_{ i })P(A|E_{ i }) } } $$

This is the Bayes’ formula, and it allows us to ‘turnaround’ conditional probabilities, i.e., we can calculate \(P(E_i|A)\)  if given information only about \(P(A|E_i)\).

Take note of the explanations given below.

1. \(P(E_j)\) are known as prior probabilities.
2. Event \(A\) is some event known to have occurred.
3. \(P(E_i|A)\) is the posterior probability.

Example: Bayes’ Formula

A Civil Engineer wishes to investigate the punctuality of electric trains by considering the number of train journeys. In the sample, 50% of trains were destined for New York, 30% for Vegas, and 20% for Washington, DC. The probabilities of a train arriving late in New York, Vegas, and Washington, DC, are 40%, 35%, and 25%, respectively. If the Engineer picks a train at random from this group, what is the probability that it would be one destined for New York?

Solution:

We are looking for \(P\text{(New York | Late)}\).

Let us define the events that are critical in our calculation.

First, \(N\) is the event “A train chosen at random would be one destined for New York.”

Secondly, \(V\) is the event “A train chosen at random would be destined for Vegas.”

And \(W\) is the event “A train chosen at random would be destined for Washington DC.”

Finally, let \(L\) be the event “A randomly chosen would arrive late.”

$$ \begin{align*} P(N|L) & =\cfrac { P(N)P(L|N) }{ P(N)P(L|N)+P(V)P(L|V)+P(W)P(L|W) } \\ & =\cfrac { 0.5×0.4 }{ 0.5× 0.4+0.3×0.35+0.2× 0.25 } \\ & =\cfrac { 0.2 }{ 0.355 } \\ & =0.5634 \\ & =56.3\% \\ \end{align*} $$

We have computed \(P(N | L)\) given only \(P(L | N)\), hence the phrase ‘turnaround conditional probability’.

Question

A chartered analyst can choose any one of three routes, A, B, or C, to get to work. The probabilities that she arrives on time using routes A, B, and C are 50%, 52%, and 60%, in that order. If she is equally likely to choose any one of the routes and arrive on time, the probability that she chose route A is closest to:

A. 30.9%.

B. 16.67%.

C. 25%.

Solution

The correct answer is A.

First, you should define the relevant events.

Let \(A\) be the event “Chooses route A.”

Let \(B\) be the event “Chooses route B.”

And let \(C\) be the event “Chooses route C.”

Lastly, define event T as “Arrives to work on time.”

Now, what we have is \(P(T | A)\), i.e., the probability that the analyst arrives on time, given that she chooses route A.

However, we want to find the turnaround probability \(P(A | T)\), i.e., the probability that the analyst chooses route A, given that she arrives on time.

This is what calls for the application of Bayes’ formula:

$$ \begin{align*} P(A|T) & =\cfrac { P(A)P(T|A) }{ P(A)P(T|A)+P(B)P(T|B)+P(C)P(T|C) } \\ & =\cfrac { \frac {1}{3} ×0.5 }{ \frac {1}{3} ×0.5+\frac {1}{3} × 0.52+\frac {1}{3} × 0.6 } \\ & =\cfrac { 0.16667 }{ 0.54 } \\ & =0.30865 \\ & =30.9\% \\ \end{align*} $$