The p-value in Hypothesis Testing

The p-value is the lowest level of significance at which we can reject a null hypothesis. It is the probability of coming up with a test statistic that would justify our rejection of a null hypothesis, assuming that the null hypothesis is indeed true.

Breaking Down the p-value

When carrying out a statistical test with a fixed value of the significance level (α), we merely compare the observed test statistic with some critical value. For example, we might “reject a H₀ using a 5% test” or “reject a H₀ at 1% significance level.” The problem with this ‘classical’ approach is that it does not give us the details about the strength of the evidence against the null hypothesis.

Determination of the p-value gives statisticians a more informative approach to hypothesis testing. The p-value is actually the lowest level at which we can reject a H₀. This means that the strength of the evidence against a H₀ increases as the p-value becomes smaller.

For one-tailed tests, the p-value is given by the probability that lies below the calculated test statistic for left-tailed tests. In the case of right-tailed tests, the probability that lies above the test statistic gives the p-value. However, if the test is two-tailed, this value is given by the sum of the probabilities in the two tails. We start by determining the probability lying below the negative value of the test statistic. After this, we add this to the probability lying above the positive value of the test statistic.

Example: p-value

θ represents the probability of obtaining a head when a coin is tossed. Assume that we tossed a coin 200 times and the head came up in 85 out of of the 200 trials. Test the following hypothesis at a 5% level of significance.

H₀: θ = 0.5

H₁: θ < 0.5

Solution

First, note that repeatedly tossing a coin follows a binomial distribution.

Our p-value will be given by P(X < 85) where X follows a binomial (200,0.5), assuming the H₀ is true.

$$ \begin{align*}
& = P \left[ Z < \frac {(85.5 – 100)}{\sqrt {50}} \right] \\
& = P(Z < -2.05) = 1 – 0.97982 = 0.02018 \\
\end{align*} $$

(We have applied the Central Limit Theorem by taking the binomial distribution as approximately normal.)

Since the probability is less than 0.05, the H₀ is extremely unlikely, and we actually have strong evidence against a H₀ that favors H₁. Therefore, clearly expressing this result, we could say:

“There is very strong evidence against the hypothesis that the coin is fair. We, therefore, conclude that the coin is biased against heads.”

Remember, failure to reject a H₀ does not mean it is true. It means there is insufficient evidence to justify the rejection of the H₀ given a certain level of significance.

Question

A CFA candidate conducts a statistical test about the mean value of a random variable X.

H₀: μ = μ₀ vs H₁: μ ≠ μ₀

She obtains a test statistic of 2.2. Given a 5% significance level, determine the p-value.

A. 1.39%.

B. 2.78.

C. 2.78%.

Solution

The correct answer is C.

$$ \text P-\text{value}= P(Z > 2.2) = 1 – P(Z < 2.2) = 1.39\% × 2 = 2.78\% $$

(We have multiplied by two since this is a two-tailed test.)

Interpretation: The p-value (2.78%) is less than the level of significance (5%). Therefore, we have sufficient evidence to reject the H₀. In fact, the evidence is so strong that we would also reject the H₀ at significance levels of 4% and 3%. However, at significance levels of 2% or 1%, we would not reject the H₀ since the p-value surpasses these values.

The p-value is the lowest level of significance at which we can reject a null hypothesis. It is the probability of coming up with a test statistic that would justify our rejection of a null hypothesis, assuming that the null hypothesis is indeed true.

Breaking Down the p-value

When carrying out a statistical test with a fixed value of the significance level (α), we merely compare the observed test statistic with some critical value. For example, we might “reject a H₀ using a 5% test” or “reject a H₀ at 1% significance level.” The problem with this ‘classical’ approach is that it does not give us the details about the strength of the evidence against the null hypothesis.

Determination of the p-value gives statisticians a more informative approach to hypothesis testing. The p-value is actually the lowest level at which we can reject a H₀. This means that the strength of the evidence against a H₀ increases as the p-value becomes smaller.

For one-tailed tests, the p-value is given by the probability that lies below the calculated test statistic for left-tailed tests. In the case of right-tailed tests, the probability that lies above the test statistic gives the p-value. However, if the test is two-tailed, this value is given by the sum of the probabilities in the two tails. We start by determining the probability lying below the negative value of the test statistic. After this, we add this to the probability lying above the positive value of the test statistic.

Example: p-value

θ represents the probability of obtaining a head when a coin is tossed. Assume that we tossed a coin 200 times and the head came up in 85 out of of the 200 trials. Test the following hypothesis at a 5% level of significance.

H₀: θ = 0.5

H₁: θ < 0.5

Solution

First, note that repeatedly tossing a coin follows a binomial distribution.

Our p-value will be given by P(X < 85) where X follows a binomial (200,0.5), assuming the H₀ is true.

$$ \begin{align*}
& = P \left[ Z < \frac {(85.5 – 100)}{\sqrt {50}} \right] \\
& = P(Z < -2.05) = 1 – 0.97982 = 0.02018 \\
\end{align*} $$

(We have applied the Central Limit Theorem by taking the binomial distribution as approximately normal.)

Since the probability is less than 0.05, the H₀ is extremely unlikely, and we actually have strong evidence against a H₀ that favors H₁. Therefore, clearly expressing this result, we could say:

“There is very strong evidence against the hypothesis that the coin is fair. We, therefore, conclude that the coin is biased against heads.”

Remember, failure to reject a H₀ does not mean it is true. It means there is insufficient evidence to justify the rejection of the H₀ given a certain level of significance.

Question

A CFA candidate conducts a statistical test about the mean value of a random variable X.

H₀: μ = μ₀ vs H₁: μ ≠ μ₀

She obtains a test statistic of 2.2. Given a 5% significance level, determine the p-value.

A. 1.39%.

B. 2.78.

C. 2.78%.

Solution

The correct answer is C.

$$ \text P-\text{value}= P(Z > 2.2) = 1 – P(Z < 2.2) = 1.39\% × 2 = 2.78\% $$

(We have multiplied by two since this is a two-tailed test.)

Interpretation: The p-value (2.78%) is less than the level of significance (5%). Therefore, we have sufficient evidence to reject the H₀. In fact, the evidence is so strong that we would also reject the H₀ at significance levels of 4% and 3%. However, at significance levels of 2% or 1%, we would not reject the H₀ since the p-value surpasses these values.