Which of the following is not true about $p$-values in hypothesis testing?

• A. If the $p$-value is less than the significance level, we reject the null hypothesis.
• B. A $p$-value represents the probability of obtaining a test statistic as extreme as, or more extreme than, the observed value, assuming the null hypothesis is true.
• C. A $p$-value tells us the probability that the null hypothesis is true.
• D. A small $p$-value indicates strong evidence against the null hypothesis.

Answer: (C)

Verified step by step guidance

1. Step 1: Understand the definition of a p-value. A p-value is the probability of obtaining a test statistic at least as extreme as the one observed, assuming the null hypothesis is true. This means it measures how compatible the data is with the null hypothesis.
2. Step 2: Review the decision rule in hypothesis testing. If the p-value is less than the chosen significance level (\$\alpha\$), we reject the null hypothesis because the observed data is unlikely under the null hypothesis.
3. Step 3: Recognize that a small p-value indicates strong evidence against the null hypothesis, suggesting that the observed data is not consistent with the null hypothesis.
4. Step 4: Identify the common misconception: a p-value does NOT tell us the probability that the null hypothesis is true. Instead, it assumes the null hypothesis is true and tells us about the probability of the observed data under that assumption.
5. Step 5: Conclude that the statement "A p-value tells us the probability that the null hypothesis is true" is not true, as it misinterprets the meaning of the p-value.