Observational Studies vs Designed Experiments: Data Collection and Confounding

Data Collection: Observational Studies vs Designed Experiments

Observational Studies

Observational studies are a fundamental method of data collection in statistics, where researchers observe subjects without manipulating any variables. These studies are often used when it is unethical or impractical to conduct experiments.

• Definition: An observational study is a study in which the researcher observes and records behavior or outcomes without influencing the subjects or variables.

• Example: In a study of cell phone usage and brain tumors, researchers followed groups of women over seven years, comparing incidence rates of brain tumors between users and non-users. No significant difference was found, and no attempt was made to influence cell phone usage.

• Key Points:

  • Researchers do not manipulate the explanatory variable.

  • Can identify associations, but cannot establish causation.

Designed Experiments

Designed experiments involve actively manipulating one or more explanatory variables to observe their effect on a response variable. This approach allows researchers to establish causation under controlled conditions.

• Definition: A designed experiment is a study in which the researcher intentionally changes the value of an explanatory variable and observes the resulting changes in the response variable.

• Example: The US National Toxicology Program exposed groups of rats to radio-frequency radiation to study the association between cell phone usage and brain tumors. The manipulation of exposure makes this a designed experiment.

• Key Points:

  • Researchers manipulate the explanatory variable.

  • Allows for claims of causation if confounding variables are controlled.

Variables in Statistical Studies

Explanatory and Response Variables

Understanding the roles of variables is essential in statistical studies. The explanatory variable is the variable that is manipulated or categorized to explain changes in the response variable.

• Explanatory Variable: The variable that is presumed to cause or explain changes in another variable.

• Response Variable: The outcome or variable that is measured to assess the effect of the explanatory variable.

• Example: In the influenza vaccine study, the explanatory variable is whether seniors received a flu shot, and the response variable is the rate of hospitalization or death from influenza or pneumonia.

Confounding and Lurking Variables

Confounding Variables

Confounding occurs when the effects of two or more explanatory variables cannot be separated, making it difficult to determine which variable is responsible for changes in the response variable.

• Definition: A confounding variable is an explanatory variable considered in a study whose effect cannot be distinguished from another explanatory variable.

• Example: In the influenza study, age, race, and health history are confounding variables because their effects may overlap with the effect of the flu shot.

Lurking Variables

Lurking variables are variables not considered in the study that nonetheless affect the response variable. They can lead to misleading associations.

• Definition: A lurking variable is an explanatory variable not included in the study but that affects the response variable.

• Example: Lifestyle choices in the influenza study are lurking variables, as they were not accounted for but may influence hospitalization rates.

Association vs Causation

Observational studies can identify associations between variables, but cannot establish causation due to potential confounding and lurking variables. Designed experiments, when properly controlled, can establish causation.

• Association: A relationship between two variables, where changes in one variable are related to changes in another.

• Causation: A relationship where one variable directly causes changes in another.

• Key Point: Observational studies are limited to claims of association, not causation.

Case Study: Television in the Bedroom and BMI

Study Description

Researchers questioned 379 twelve-year-old adolescents to determine if having a television in the bedroom is associated with higher levels of body mass index (BMI). The study found that adolescents with a TV in their bedroom had significantly higher BMI than those without.

• Explanatory Variable: Presence of a TV in the bedroom

• Response Variable: Level of BMI

• Study Type: Observational study

• Lurking Variables: Socioeconomic status, lifestyle, physical activity, diet

• Statistical Significance: The results remained significant after adjustment for socioeconomic status, suggesting the association is not solely due to this variable.

• Conclusion: The study shows an association, not causation, between having a TV in the bedroom and higher BMI.

Summary Table: Observational Studies vs Designed Experiments

Study Type	Manipulation of Variables	Can Claim Causation?	Example
Observational Study	No	No, only association	Cell phone usage and brain tumors in women; TV in bedroom and BMI
Designed Experiment	Yes	Yes, if confounding is controlled	Rats exposed to radio-frequency radiation

Key Formulas

• Relative Risk: Used to compare the risk of an outcome between two groups. $RR = \frac{P_{exposed}}{P_{unexposed}}$

• Statistical Significance: Often assessed using p-values in hypothesis testing. $p = P(\text{observed result} | \text{null hypothesis is true})$

Conclusion

Understanding the distinction between observational studies and designed experiments is crucial for interpreting statistical results. Observational studies are valuable for identifying associations, especially when experiments are unethical or impractical, but they are limited by confounding and lurking variables. Designed experiments, when feasible, provide stronger evidence for causation.