In the context of hypothesis testing, what does it mean to say a result is statistically significant at significance level $\alpha$?

• A. The alternative hypothesis is proven true with certainty.
• B. The p-value is less than or equal to $\alpha$, so the null hypothesis is rejected.
• C. The effect size is large in practical terms, regardless of the p-value.
• D. The p-value is greater than $\alpha$, so the null hypothesis is rejected.

Answer: (B)

Verified step by step guidance

1. Understand that in hypothesis testing, the significance level, denoted by \(\alpha\), is a threshold set by the researcher before conducting the test. It represents the maximum probability of making a Type I error, which is rejecting the null hypothesis when it is actually true.
2. Recognize that the p-value is the probability of obtaining test results at least as extreme as the observed results, assuming the null hypothesis is true.
3. Compare the p-value to the significance level \(\alpha\). If the p-value is less than or equal to \(\alpha\), it indicates that the observed data is sufficiently unlikely under the null hypothesis.
4. Conclude that when the p-value \(\leq \alpha\), the null hypothesis is rejected in favor of the alternative hypothesis. This is what is meant by the result being statistically significant at level \(\alpha\).
5. Note that statistical significance does not prove the alternative hypothesis with certainty, nor does it necessarily imply a large or practically important effect size.