The p-value in Hypothesis Testing

The p-value in Hypothesis Testing

The p-value is the lowest level of significance at which we can reject a null hypothesis. It is the probability of coming up with a test statistic that would justify our rejection of a null hypothesis, assuming that the null hypothesis is indeed true.

Breaking Down the p-value

When carrying out a statistical test with a fixed value of the significance level (α), we merely compare the observed test statistic with some critical value. For example, we might “reject a H0 using a 5% test” or “reject a H0 at 1% significance level.” The problem with this ‘classical’ approach is that it does not give us the details about the strength of the evidence against the null hypothesis.

Determination of the p-value gives statisticians a more informative approach to hypothesis testing. The p-value is actually the lowest level at which we can reject a H0. This means that the strength of the evidence against a H0 increases as the p-value becomes smaller.

For one-tailed tests, the p-value is given by the probability that lies below the calculated test statistic for left-tailed tests. In the case of right-tailed tests, the probability that lies above the test statistic gives the p-value. However, if the test is two-tailed, this value is given by the sum of the probabilities in the two tails. We start by determining the probability lying below the negative value of the test statistic. After this, we add this to the probability lying above the positive value of the test statistic.

Example: p-value

θ represents the probability of obtaining a head when a coin is tossed. Assume that we tossed a coin 200 times and the head came up in 85 out of of the 200 trials. Test the following hypothesis at a 5% level of significance.

H0: θ = 0.5

H1: θ < 0.5

Solution

First, note that repeatedly tossing a coin follows a binomial distribution.

Our p-value will be given by P(X < 85) where X follows a binomial (200,0.5), assuming the H0 is true.

$$ \begin{align*}
& = P \left[ Z < \frac {(85.5 – 100)}{\sqrt {50}} \right] \\
& = P(Z < -2.05) = 1 – 0.97982 = 0.02018 \\
\end{align*} $$

(We have applied the Central Limit Theorem by taking the binomial distribution as approximately normal.)

Since the probability is less than 0.05, the H0 is extremely unlikely, and we actually have strong evidence against a H0 that favors H1. Therefore, clearly expressing this result, we could say:

“There is very strong evidence against the hypothesis that the coin is fair. We, therefore, conclude that the coin is biased against heads.”

Remember, failure to reject a H0 does not mean it is true. It means there is insufficient evidence to justify the rejection of the H0 given a certain level of significance.

Question

A CFA candidate conducts a statistical test about the mean value of a random variable X.

H0: μ = μ0 vs H1: μ μ0

She obtains a test statistic of 2.2. Given a 5% significance level, determine the p-value.

A. 1.39%.

B. 2.78.

C. 2.78%.

Solution

The correct answer is C.

$$ \text P-\text{value}= P(Z > 2.2) = 1 – P(Z < 2.2) = 1.39\% × 2 = 2.78\% $$

(We have multiplied by two since this is a two-tailed test.)

p-value

Interpretation: The p-value (2.78%) is less than the level of significance (5%). Therefore, we have sufficient evidence to reject the H0. In fact, the evidence is so strong that we would also reject the H0 at significance levels of 4% and 3%. However, at significance levels of 2% or 1%, we would not reject the H0 since the p-value surpasses these values.


The p-value is the lowest level of significance at which we can reject a null hypothesis. It is the probability of coming up with a test statistic that would justify our rejection of a null hypothesis, assuming that the null hypothesis is indeed true.

Breaking Down the p-value

When carrying out a statistical test with a fixed value of the significance level (α), we merely compare the observed test statistic with some critical value. For example, we might “reject a H0 using a 5% test” or “reject a H0 at 1% significance level.” The problem with this ‘classical’ approach is that it does not give us the details about the strength of the evidence against the null hypothesis.

Determination of the p-value gives statisticians a more informative approach to hypothesis testing. The p-value is actually the lowest level at which we can reject a H0. This means that the strength of the evidence against a H0 increases as the p-value becomes smaller.

For one-tailed tests, the p-value is given by the probability that lies below the calculated test statistic for left-tailed tests. In the case of right-tailed tests, the probability that lies above the test statistic gives the p-value. However, if the test is two-tailed, this value is given by the sum of the probabilities in the two tails. We start by determining the probability lying below the negative value of the test statistic. After this, we add this to the probability lying above the positive value of the test statistic.

Example: p-value

θ represents the probability of obtaining a head when a coin is tossed. Assume that we tossed a coin 200 times and the head came up in 85 out of of the 200 trials. Test the following hypothesis at a 5% level of significance.

H0: θ = 0.5

H1: θ < 0.5

Solution

First, note that repeatedly tossing a coin follows a binomial distribution.

Our p-value will be given by P(X < 85) where X follows a binomial (200,0.5), assuming the H0 is true.

$$ \begin{align*}
& = P \left[ Z < \frac {(85.5 – 100)}{\sqrt {50}} \right] \\
& = P(Z < -2.05) = 1 – 0.97982 = 0.02018 \\
\end{align*} $$

(We have applied the Central Limit Theorem by taking the binomial distribution as approximately normal.)

Since the probability is less than 0.05, the H0 is extremely unlikely, and we actually have strong evidence against a H0 that favors H1. Therefore, clearly expressing this result, we could say:

“There is very strong evidence against the hypothesis that the coin is fair. We, therefore, conclude that the coin is biased against heads.”

Remember, failure to reject a H0 does not mean it is true. It means there is insufficient evidence to justify the rejection of the H0 given a certain level of significance.

Question

A CFA candidate conducts a statistical test about the mean value of a random variable X.

H0: μ = μ0 vs H1: μ μ0

She obtains a test statistic of 2.2. Given a 5% significance level, determine the p-value.

A. 1.39%.

B. 2.78.

C. 2.78%.

Solution

The correct answer is C.

$$ \text P-\text{value}= P(Z > 2.2) = 1 – P(Z < 2.2) = 1.39\% × 2 = 2.78\% $$

(We have multiplied by two since this is a two-tailed test.)

p-value

Interpretation: The p-value (2.78%) is less than the level of significance (5%). Therefore, we have sufficient evidence to reject the H0. In fact, the evidence is so strong that we would also reject the H0 at significance levels of 4% and 3%. However, at significance levels of 2% or 1%, we would not reject the H0 since the p-value surpasses these values.


Shop CFA® Exam Prep

Offered by AnalystPrep

Featured Shop FRM® Exam Prep Learn with Us

    Subscribe to our newsletter and keep up with the latest and greatest tips for success
    Shop Actuarial Exams Prep Shop Graduate Admission Exam Prep


    Sergio Torrico
    Sergio Torrico
    2021-07-23
    Excelente para el FRM 2 Escribo esta revisión en español para los hispanohablantes, soy de Bolivia, y utilicé AnalystPrep para dudas y consultas sobre mi preparación para el FRM nivel 2 (lo tomé una sola vez y aprobé muy bien), siempre tuve un soporte claro, directo y rápido, el material sale rápido cuando hay cambios en el temario de GARP, y los ejercicios y exámenes son muy útiles para practicar.
    diana
    diana
    2021-07-17
    So helpful. I have been using the videos to prepare for the CFA Level II exam. The videos signpost the reading contents, explain the concepts and provide additional context for specific concepts. The fun light-hearted analogies are also a welcome break to some very dry content. I usually watch the videos before going into more in-depth reading and they are a good way to avoid being overwhelmed by the sheer volume of content when you look at the readings.
    Kriti Dhawan
    Kriti Dhawan
    2021-07-16
    A great curriculum provider. James sir explains the concept so well that rather than memorising it, you tend to intuitively understand and absorb them. Thank you ! Grateful I saw this at the right time for my CFA prep.
    nikhil kumar
    nikhil kumar
    2021-06-28
    Very well explained and gives a great insight about topics in a very short time. Glad to have found Professor Forjan's lectures.
    Marwan
    Marwan
    2021-06-22
    Great support throughout the course by the team, did not feel neglected
    Benjamin anonymous
    Benjamin anonymous
    2021-05-10
    I loved using AnalystPrep for FRM. QBank is huge, videos are great. Would recommend to a friend
    Daniel Glyn
    Daniel Glyn
    2021-03-24
    I have finished my FRM1 thanks to AnalystPrep. And now using AnalystPrep for my FRM2 preparation. Professor Forjan is brilliant. He gives such good explanations and analogies. And more than anything makes learning fun. A big thank you to Analystprep and Professor Forjan. 5 stars all the way!
    michael walshe
    michael walshe
    2021-03-18
    Professor James' videos are excellent for understanding the underlying theories behind financial engineering / financial analysis. The AnalystPrep videos were better than any of the others that I searched through on YouTube for providing a clear explanation of some concepts, such as Portfolio theory, CAPM, and Arbitrage Pricing theory. Watching these cleared up many of the unclarities I had in my head. Highly recommended.