Which of the following is not true about $p$-values in hypothesis testing?

• A. A $p$-value represents the probability of obtaining a test statistic as extreme as, or more extreme than, the observed value, assuming the null hypothesis is true.
• B. If the $p$-value is less than the significance level, we reject the null hypothesis.
• C. A small $p$-value indicates strong evidence against the null hypothesis.
• D. A $p$-value tells us the probability that the null hypothesis is true.

Answer: (D)

Verified step by step guidance

1. Step 1: Understand the definition of a p-value. A p-value is the probability of obtaining a test statistic at least as extreme as the one observed, assuming the null hypothesis is true. This means it measures how compatible the data is with the null hypothesis.
2. Step 2: Recognize the decision rule in hypothesis testing. If the p-value is less than the chosen significance level (\$\alpha\$), we reject the null hypothesis because the observed data is unlikely under the null hypothesis.
3. Step 3: Interpret what a small p-value means. A small p-value indicates strong evidence against the null hypothesis, suggesting that the observed data is unlikely to have occurred if the null hypothesis were true.
4. Step 4: Identify the incorrect statement. The statement that a p-value tells us the probability that the null hypothesis is true is incorrect because the p-value assumes the null hypothesis is true and does not provide the probability of the hypothesis itself.
5. Step 5: Summarize that p-values do not measure the probability of the null hypothesis being true or false; rather, they measure the probability of the observed data under the assumption that the null hypothesis is true.