Scope of Inference in Statistical Studies

Introduction

Understanding the scope of inference is crucial in statistics, as it determines what conclusions can be drawn from a study. The ability to generalize results to a larger population and to establish causation depends on how the study is designed, specifically whether random sampling and random assignment are used.

Random Sampling and Random Assignment

Definitions

• Random Sampling: Selecting individuals from a population in such a way that every individual has an equal chance of being chosen. This allows results to be generalized to the population from which the sample was drawn.

• Random Assignment: Assigning study participants to different groups (such as treatment and control) using a random process. This allows for causal conclusions about the effect of the treatment.

Key Points

• Generalization (Inference about Population): Possible only if random sampling is used.

• Causation (Inference about Cause and Effect): Possible only if random assignment is used.

Table: Scope of Inference Based on Study Design

The following table summarizes when it is appropriate to make inferences about the population and/or causation based on whether random sampling and random assignment are used:

Types of Studies

Observational Study vs. Experiment

• Observational Study: Researchers observe subjects and measure variables of interest without assigning treatments. No random assignment is used, so causation cannot be established.

• Experiment: Researchers actively impose treatments on subjects and use random assignment to groups. This allows for causal conclusions.

Example: SAT Prep Course Study Designs

Suppose we want to determine if taking an SAT prep course improves SAT scores. Several study designs are possible:

• Random Sample, No Random Assignment: Select a random sample of students and observe who took the course. Can generalize to the population, but cannot infer causation.

• No Random Sample, Random Assignment: Use a convenience sample (e.g., one class) and randomly assign students to take or not take the course. Can infer causation for the sample, but cannot generalize to the population.

• Random Sample and Random Assignment: Select a random sample and randomly assign to treatment. Can both generalize and infer causation.

Conclusion Choice Bank

• Association: There is a relationship between the treatment and the outcome, but causation is not established.

• Causation: The treatment causes changes in the outcome, which can only be concluded with random assignment.

• Generalization: Results can be extended to the population only if random sampling is used.

Application Example: iPhone Battery Life Study

Study Description

A random sample of 20 new iPhones is split into two groups. One group is set to low screen brightness, the other to high. Battery life is measured for each phone.

• Type of Study: Experiment (random assignment to treatment groups)

• Explanatory Variable: Screen brightness (low or high)

• Response Variable: Battery life (time until battery is dead)

• Random Sample Used: Yes

• Random Assignment Used: Yes

• Conclusion: For all brand new iPhones from this store, setting screen brightness to low causes increased battery life.

Summary

• Random sampling allows generalization to the population.

• Random assignment allows inference about causation.

• Both are needed to generalize causal conclusions to the population.