Question 1

Which of the following statements about p-values is TRUE according to the ASA statement?

Accepted Answer

A p-value indicates how incompatible the data are with a specified statistical model.

Question 2

A researcher finds a statistically significant difference between two groups with a p-value of 0.03. The difference in means is from 75% to 76%. What is the best interpretation of this result?

Accepted Answer

The effect may not be important, as statistical significance does not imply practical importance.

Question 3

Which of the following is a misconception about non-significant results?

Accepted Answer

A non-significant result means the effect size is zero.

Question 4

What is the main advantage of reporting effect sizes in addition to p-values?

Accepted Answer

Effect sizes provide a standardized measure of the magnitude of an effect, allowing for better interpretation.

Question 5

Which of the following practices is an example of p-hacking?

Accepted Answer

Stopping data collection once a significant p-value is obtained.

Question 6

A meta-analysis is best described as:

Accepted Answer

A statistical technique that combines effect sizes from multiple studies to estimate a population effect.

Question 7

Which formula correctly represents Cohen's d for the difference between two group means?

Accepted Answer

$$d = \frac{M_1$ - M_2$}{s_p}$$

Question 8

Which of the following best describes the purpose of preregistration in research?

Accepted Answer

To make the research process transparent and reduce questionable research practices.

Question 9

If a 95% confidence interval for a mean difference includes zero, what does this imply?

Accepted Answer

There is not enough evidence to conclude a significant difference exists.

Question 10

Which of the following is NOT a recommended solution to the problems associated with Null Hypothesis Significance Testing (NHST)?

Accepted Answer

Engaging in p-hacking to ensure significant results.

Question 11

A study finds $$r = 0.3$$ between two variables. According to the guidelines, how much variance is explained by this effect?

Accepted Answer

9%

Question 12

Which of the following is a key benefit of Bayesian estimation compared to NHST?

Accepted Answer

It allows for direct probability statements about hypotheses.

Question 13

What is the main problem with 'all-or-nothing' thinking in NHST?

Accepted Answer

It leads to the dismissal of potentially meaningful effects that are not statistically significant.

Question 14

Which of the following is an example of a researcher's 'degree of freedom' that could lead to questionable research practices?

Accepted Answer

Choosing which statistical model to fit after seeing the data.

Question 15

Which of the following best describes the relationship between sample size and statistical significance?

Accepted Answer

Larger sample sizes can make even small effects statistically significant.

Question 16

Which of the following is TRUE about effect sizes?

Accepted Answer

Effect sizes are standardized and allow comparison across studies.

Question 17

What is the main goal of meta-analysis in scientific research?

Accepted Answer

To estimate the population effect by combining results from multiple studies.

Question 18

Which of the following is a correct interpretation of a p-value of 0.04?

Accepted Answer

The observed data are unlikely under the null hypothesis, given the chosen significance level.

Question 19

Which of the following is NOT a benefit of Bayesian approaches in statistics?

Accepted Answer

They encourage black-and-white thinking about results.

Question 20

Which of the following is the best reason for using open/shared data in research?

Accepted Answer

To allow other researchers to verify and build upon findings, increasing transparency.

Null Hypothesis Significance Testing, Effect Sizes, and Modern Issues in Statistical Inference

Study Guide - Practice Questions

Study Guide - Flashcards