Chapter 4: Gathering Data

Introduction to Study Design

Study design is a foundational concept in statistics, focusing on how data is collected to ensure valid and reliable results. Proper study design helps minimize bias and allows for meaningful inferences about populations.

Population and Sample

Definitions and Importance

• Population: The entire group of subjects or items of interest in a study.

• Sample: A subset of the population, selected for actual data collection because it is often impractical to measure the whole population.

• Statistics are used to make inferences about the population based on sample data.

Types of Studies

Observational Studies

In an observational study, researchers observe and measure variables of interest without assigning treatments to the subjects. The goal is to identify associations between variables.

• Explanatory variable: The variable that is thought to explain or cause changes in the response variable (e.g., hourly use of cell phone).

• Response variable: The outcome measured (e.g., presence of a brain tumor).

• Sample survey: A type of observational study where a sample is interviewed to collect data.

• Census: A survey that attempts to include every member of the population.

Experimental Studies

In an experiment, researchers assign subjects to specific conditions (treatments) and observe the outcomes. This design allows for the establishment of cause-and-effect relationships.

• Treatment: The specific experimental condition imposed on subjects, corresponding to values of the explanatory variable.

Comparing Experimental and Observational Studies

• Experiments reduce the influence of lurking variables and allow for causal conclusions.

• Observational studies can only identify associations, not causation.

• Experiments may not always be feasible due to ethical or practical constraints.

Sampling Methods

Sampling Frame and Sampling Design

The sampling design is the method used to select the sample from the population. The sampling frame is the list of all subjects in the population from which the sample is drawn. Ideally, every subject should have an equal chance of being selected.

Simple Random Sample (SRS)

A simple random sample (SRS) of size n is one in which every possible sample of that size has the same chance of being selected. SRS is the gold standard for obtaining a representative sample.

• Random numbers can be generated using tables or computer algorithms.

Steps for Selecting a Simple Random Sample

1. Number the subjects in the sampling frame using numbers of the same length.

2. Select numbers of that length from a table of random numbers or a random number generator.

3. Include in the sample those subjects whose numbers match the selected random numbers.

Margin of Error

The margin of error quantifies the uncertainty in estimates from a sample survey. It indicates how close the sample estimate is likely to be to the true population value.

• For a SRS of n subjects, the margin of error is approximately:

• Example: If a survey reports a margin of error of ±3%, the true population percentage is likely within 3% of the sample percentage.

Poor Sampling Methods

• Convenience sample: A sample that is easy to obtain but unlikely to be representative (e.g., surveying people at a mall).

• Volunteer sample: Subjects volunteer to participate, often leading to bias as volunteers may not represent the population.

Types of Bias in Sample Surveys

• Sampling bias: Systematic favoring of certain parts of the population due to the sampling method (e.g., undercoverage).

• Nonresponse bias: Some sampled subjects cannot be reached or refuse to participate.

• Response bias: Subjects give incorrect responses or questions are misleading.

• A large sample does not guarantee an unbiased sample if the sampling method is flawed.

Key Parts of a Sample Survey

• Identify the population of interest.

• Construct a sampling frame listing all subjects in the population.

• Use a random sampling design to select subjects.

• Be cautious of bias due to nonrandom samples.

Designing Good Experiments

Elements of an Experiment

• Experimental units: The subjects or entities measured in an experiment.

• Treatment: The specific condition imposed on the subjects.

• Explanatory variable (factor): The variable defining groups to be compared.

• Response variable: The outcome measured to assess the effect of the treatment.

Principles of Good Experimental Design

• Control/Comparison group: Allows analysis of the treatment's effectiveness, often using a placebo or standard treatment.

• Randomization: Subjects are randomly assigned to treatments to eliminate bias and balance groups on known and unknown variables.

• Replication: Assigning multiple subjects to each treatment to reduce the impact of random variation and increase confidence in results.

Blinding in Experiments

• Blinding: Subjects are unaware of which treatment they receive.

• Double-blinding: Neither subjects nor investigators know the treatment assignments, reducing response and experimenter bias.

Statistical Significance

• A result is statistically significant if the observed difference is unlikely to have occurred by chance alone.

• Practical significance should also be considered—results should be meaningful in real-world terms, not just statistically significant.

Generalizing Results

• Results should only be generalized to the population represented by the study.

• Care must be taken to ensure the sample and experimental conditions reflect the target population and context.

Lesson Summary

• Two primary methods for gathering data: observational studies and experiments.

• Observational studies can identify associations but not causation.

• Randomization (SRS) is essential for minimizing the effects of lurking variables and bias.

• Awareness of potential biases is crucial in study design.

• Experiments assign treatments and often use control groups to compare outcomes.