Chapter 4: Gathering Data – Study Notes for Statistics Students

Chapter 4: Gathering Data

Section 4.1: Experimental and Observational Studies

Gathering data is a fundamental step in statistics, as the quality and method of data collection directly impact the validity of statistical inference. There are two primary types of studies: observational and experimental.

• Observational Study: The researcher observes values of the response and explanatory variables for sampled subjects without imposing any treatment.

• Experimental Study (Experiment): The researcher assigns subjects to specific experimental conditions (treatments) and observes outcomes on the response variable.

• Response Variable: Measures the outcome of a study.

• Explanatory Variable: May explain or influence changes in the response variable.

• Population: The entire group of individuals of interest.

• Sample: A subset of the population from which information is collected.

Example: A Gallup poll surveyed 1,015 adults across the U.S. to measure satisfaction with K-12 education. The sample is the 1,015 adults, the population is all U.S. adults, and the statistic is the 43% satisfaction rate.

Advantages of Experiments: Experiments reduce the influence of lurking variables and allow for stronger causal inference compared to observational studies.

Section 4.2: Good and Poor Ways to Sample

The method of sampling is crucial for obtaining representative data. Randomization is key to avoiding bias.

• Sampling Design: The plan for choosing a sample from the population.

• Sampling Frame: The list of units from which the sample is chosen; may not perfectly match the population.

• Bias: Systematic favoring of certain outcomes due to study design.

Simple Random Sampling

Each set of n elements in the population has an equal chance of being selected. This method is often implemented using random number generators.

• Sampling Without Replacement: Each unit can only be selected once.

Methods of Collecting Data

• Personal interview

• Telephone interview

• Self-administered questionnaire

Sources of Bias

• Volunteer Sample: People choose themselves to respond; often biased.

• Convenience Sampling: Choosing the easiest members to reach; may be necessary but not ideal.

• Undercoverage: Some groups are left out of the sampling process.

• Nonresponse: Individuals chosen for the sample do not respond.

• Response Bias: Respondents provide incorrect answers, possibly due to question wording or interviewer influence.

Wording Effects: The phrasing and order of questions can significantly affect survey responses.

Summary of Bias Types: Sampling bias, nonresponse bias, and response bias.

Random Sampling: Eliminates bias and allows for trustworthy inference using probability laws.

Large Sample Size: Only improves accuracy if the sample is unbiased; a large biased sample is not useful.

Section 4.3: Good and Poor Ways to Experiment

Experiments are designed to study whether treatments cause changes in the response variable. Proper experimental design is essential for valid results.

• Experimental Units: Individuals on which the experiment is conducted.

• Factors: Explanatory variables in an experiment.

• Levels: Specific values of a factor.

• Treatments: Specific experimental conditions applied to units.

Example: Studying fabric durability with different water temperatures and cleansing agents involves multiple factors and treatments.

• Control Group: Used for comparison, often receives a placebo.

• Randomized Comparative Experiment: Uses randomization to balance groups and reduce bias.

• Completely Randomized Design: All units are randomly allocated among treatments.

Principles of Experimental Design:

1. Control

2. Randomization

3. Replication

• Statistical Significance: An effect so large it would rarely occur by chance.

• Placebo Effect: Response due to expectations, not the treatment itself.

• Blind Experiment: Participants do not know which treatment they receive.

• Double-Blind Experiment: Neither participants nor evaluators know which treatment is assigned.

Generalizing Results: Results may not generalize if the sample is not randomly selected from the population.

Section 4.4: Other Ways to Conduct Experimental and Nonexperimental Studies

There are several probability sampling methods beyond simple random sampling, each with specific advantages and disadvantages.

Stratified Random Sample

The population is divided into strata (groups), and a simple random sample is taken from each stratum. Useful when response variable values differ across strata.

Cluster Random Sample

The population is divided into clusters, and a simple random sample of clusters is selected. All subjects in chosen clusters are included in the sample. Useful when a sampling frame is unavailable or costs are high.

• Probability Sample: Each member of the population has a known chance of being selected.

Multistage Sample Design: Involves selecting clusters and then individuals within clusters; commonly used in national surveys.

Advantages and Disadvantages: Stratified sampling ensures representation of each group but requires a sampling frame. Cluster sampling is cost-effective but may require larger sample sizes for precision.

Retrospective and Prospective Observational Studies

Observational studies can be classified based on the timing of data collection.

• Retrospective Study: Looks into the past, often used in medical research (case-control studies).

• Prospective Study: Identifies a cohort and observes them in the future (cohort studies).

• Sample Survey: Takes a cross-section of the population at the current time (cross-sectional study).

Case-Control Study: Compares subjects with a response outcome of interest (cases) to those without (controls) on explanatory variables.

Establishing Causation: Observational studies rarely establish causation definitively due to potential lurking variables. Multiple lines of evidence and experiments strengthen causal claims.

Multifactor Experiments and Block Designs

Experiments often involve multiple factors. Designs such as matched pairs and block designs help control for variability and reduce bias.

• Matched Pairs Design: Compares two treatments using closely matched pairs; treatments are randomly assigned within each pair.

• Crossover Design: Subjects switch treatments during the experiment, reducing bias.

• Block Design: Groups subjects known to be similar in some way; random assignment occurs within each block.

Example: Comparing responses to advertisements among men and women using block design ensures differences between groups are accounted for.

Blinding and Double-Blinding: Used to prevent bias in experiments by concealing treatment assignments from subjects and evaluators.

Summary Table: Sampling Methods

Method	Description	Advantages
Simple random sample	Each possible sample is equally likely	Sample tends to be a good reflection of the population
Cluster random sample	Identify clusters of subjects, take simple random sample of the clusters	Do not need a sampling frame of subjects, less expensive to implement
Stratified random sample	Divide population into groups (strata), take simple random sample from each stratum	Ensures enough subjects in each group that you want to compare

Key Formulas

• Margin of Sampling Error: $\pm z^* \frac{\sqrt{p(1-p)}}{\sqrt{n}}$

• Sample Proportion: $\hat{p} = \frac{x}{n}$

Additional info: The notes expand on textbook definitions and provide practical examples for each sampling and experimental method, ensuring students understand both theoretical and applied aspects of data collection in statistics.