Observational Studies vs. Designed Experiments: Identifying Variables and Understanding Confounding

Chapter 1.2: Observational Studies vs. Designed Experiments

Distinguishing Between Observational Studies and Designed Experiments

Understanding the difference between observational studies and designed experiments is fundamental in statistics, as it determines the strength of conclusions that can be drawn about relationships between variables.

• Observational Study: The researcher observes and measures characteristics of interest without attempting to influence outcomes. No treatment is applied by the researcher.

• Designed Experiment: The researcher assigns individuals to groups, deliberately changes the value of an explanatory variable, and records the response variable for each group. This allows for control over variables and the possibility of establishing causation.

Example:

• Study A (Cellular Phones and Brain Tumors): Researchers followed 791,710 women over 7 years, comparing brain tumor incidence between mobile phone users and non-users. This is an observational study because the researchers did not assign phone usage.

• Study B (Cellular Phones and Brain Tumors in Rats): Rats were assigned to control and exposure groups, with exposure to different types of radio-frequency radiation (RFR). This is a designed experiment because the researchers controlled the exposure.

Identifying Explanatory, Response, and Lurking Variables

In any study, it is crucial to distinguish between the variables involved:

• Explanatory Variable: The variable that is manipulated or categorized to observe its effect on the response variable.

• Response Variable: The outcome or variable measured to assess the effect of the explanatory variable.

• Lurking Variable: A variable not considered in the study that may influence both the explanatory and response variables, potentially confounding the results.

Example:

• In both cell phone studies, the explanatory variable is the level of cell phone usage or RFR exposure, and the response variable is whether or not brain tumors developed.

Practice Identifying Study Types and Variables

• Question A: Flu Shots and Seniors - Type: Observational study (seniors chose whether to get vaccinated; no assignment by researchers). - Explanatory Variable: Whether the senior received a flu shot. - Response Variable: Hospitalization or death from pneumonia or influenza.

• Question B: Television in the Bedroom - Type: Observational study (researchers observed existing conditions; no assignment). - Explanatory Variable: Presence of a TV in the bedroom. - Response Variable: Body Mass Index (BMI) of adolescents.

• Question C: Traditional vs. Reform Math Methods - Type: Designed experiment (students were randomly assigned to teaching methods). - Explanatory Variable: Teaching method (traditional or reform). - Response Variable: Achievement test scores after one year.

Lurking Variables and Confounding

Lurking variables can obscure the true relationship between explanatory and response variables, leading to confounding.

• Lurking Variable: An unmeasured variable that influences both the explanatory and response variables.

• Confounding: Occurs when the effects of two or more explanatory variables on the response variable cannot be separated.

Example:

• In the TV and BMI study, a possible lurking variable is socioeconomic status, which may affect both the likelihood of having a TV in the bedroom and BMI.

• In the influenza study, possible lurking variables include age, health status, or mobility of the seniors.

Association vs. Causation

It is important to recognize the limitations of observational studies:

• Observational studies can identify associations but cannot establish causation due to potential confounding by lurking variables.

• Designed experiments allow for control of variables and can provide evidence for cause-and-effect relationships.

Key Point: Observational studies do not allow a researcher to claim causation, only association. Designed experiments are necessary to establish causality.

Summary Table: Observational Study vs. Designed Experiment

Feature	Observational Study	Designed Experiment
Researcher Control	No manipulation; observes existing conditions	Researcher assigns treatments or groups
Can Establish Causation?	No (only association)	Yes (if well-designed)
Susceptible to Confounding?	Yes, due to lurking variables	Less so, due to randomization and control
Example	Surveying TV presence and BMI	Randomly assigning teaching methods

Applications and Interpretation

• When reading study results, always consider whether the study design allows for causal conclusions.

• Look for possible lurking variables that may confound the results, especially in observational studies.

• When researchers state that results are "significant after adjustment for socioeconomic status," it means statistical methods were used to control for this lurking variable, but other unmeasured variables may still exist.

Additional info:

• Randomization in experiments helps to balance lurking variables across groups, reducing confounding.

• Statistical methods such as regression can adjust for known lurking variables, but cannot account for unknown or unmeasured ones.