Chapter 9: Sample Surveys – Principles and Pitfalls

Sample Surveys: Foundations and Key Concepts

Introduction to Sample Surveys

Sample surveys are essential tools in statistics for making inferences about a population based on data collected from a subset, or sample, of that population. This chapter explores the principles of sampling, the importance of randomness, and common pitfalls to avoid in survey design.

Major Ideas in Sampling

Examining a Part of the Whole

• Population: The entire group of individuals or instances about whom we hope to learn.

• Sample: A smaller group selected from the population for study.

• Sampling: The process of selecting a sample to represent the population. For example, tasting a spoonful of soup to judge the whole pot.

• Sample Survey: An investigation that asks questions of a sample to learn about the population (e.g., opinion polls).

• Representativeness: A sample is representative if its statistics accurately reflect the population parameters.

Randomization

• Randomization: The use of chance to select a sample, protecting against bias from known and unknown factors.

• Ensures that, on average, the sample resembles the population.

Sample Size

• Sample Size: The number of individuals in a sample is more important than the fraction of the population sampled.

• Larger samples generally yield more precise estimates of population parameters.

Census

• Census: A special sample that includes every individual in the population.

• Often impractical due to cost, time, and difficulty in reaching all individuals.

Populations, Parameters, and Statistics

Definitions and Notation

• Parameter: A numerical summary of a population (denoted by Greek letters, e.g., $\mu$, $\sigma$, $\rho$, $\beta$, $p$).

• Statistic: A numerical summary of a sample (denoted by Latin letters, e.g., $\bar{x}$, $s$, $r$, $b$, $\hat{p}$).

• We use statistics to estimate parameters.

Name	Sample Statistic	Population Parameter
Mean	$\bar{x}$	$\mu$
Standard deviation	$s$	$\sigma$
Correlation	$r$	$\rho$
Regression coefficient	$b$	$\beta$
Proportion	$\hat{p}$	$p$

Sampling Methods

Simple Random Sample (SRS)

A simple random sample is one in which every possible group of n individuals has an equal chance of being selected. This is the gold standard for sampling methods.

• Requires a sampling frame: a list of all individuals in the population.

• Random numbers or technology can be used to select the sample.

• Sample-to-sample differences are called sampling variability.

Stratified Sampling

Stratified sampling divides the population into homogeneous groups (strata) and selects a random sample from each stratum. This method increases precision and allows for subgroup analysis.

• Reduces sampling variability.

• Allows for different sampling methods in each stratum.

• Provides estimates for each subgroup as well as the whole population.

Cluster and Multistage Sampling

Cluster sampling divides the population into clusters, randomly selects some clusters, and samples all individuals within those clusters. Multistage sampling combines several sampling methods, often using clusters and stratification.

• Clusters should be similar to the population as a whole.

• Multistage sampling is common in large-scale surveys.

Systematic Sampling

Systematic sampling selects individuals at regular intervals from a randomly chosen starting point. It is efficient and can be representative if the list order is unrelated to the variable of interest.

• Less expensive than SRS.

• Requires justification that the order does not introduce bias.

Who’s Who in a Survey

Defining Groups in a Survey

• Population of Interest: The group about which we want to draw conclusions.

• Sampling Frame: The list from which the sample is drawn (may differ from the population).

• Target Sample: The individuals intended to be measured.

• Sample (Respondents): The individuals who actually provide data.

• Each step can introduce constraints and potential biases.

Designing a Valid Survey

Best Practices

• Clearly define what you want to know and from whom.

• Use an appropriate sampling frame.

• Design survey instruments carefully to avoid measurement errors.

• Ask specific, quantitative questions when possible.

• Pilot the survey to identify potential issues.

Question and Answer Design

• Phrase questions and answers clearly and neutrally.

• Offer choices rather than open-ended responses when possible.

Common Sampling Mistakes and Biases

Types of Bias

• Voluntary Response Bias: Occurs when individuals choose to participate, often leading to unrepresentative samples.

• Convenience Sampling: Involves sampling individuals who are easiest to reach, which may not represent the population.

• Bad Sampling Frame: When the list used to draw the sample does not match the population of interest.

• Undercoverage: Some groups are not adequately represented in the sample.

• Nonresponse Bias: When individuals selected for the sample do not respond.

• Response Bias: When survey design or interviewer behavior influences responses.

How to Avoid Bias

• Use random sampling methods.

• Pilot surveys to detect issues.

• Report sampling methods in detail.

• Be vigilant for biases at every stage of survey design and implementation.

Summary Table: Sampling Methods

Method	Description	Advantages	Disadvantages
Simple Random Sample (SRS)	Every group of n individuals has equal chance of selection	Unbiased, easy to analyze	May be impractical for large populations
Stratified Sampling	Population divided into strata, random sample from each	Reduces variability, allows subgroup analysis	Requires knowledge of strata
Cluster Sampling	Population divided into clusters, some clusters sampled entirely	Efficient, practical	Clusters may not represent population well
Systematic Sampling	Every kth individual selected after random start	Simple, cost-effective	Can be biased if order is related to variable
Convenience Sampling	Sample those easiest to reach	Easy, inexpensive	Highly biased, not representative

Key Takeaways

• Sampling allows us to make inferences about populations without examining every individual.

• Randomization is crucial for representativeness and unbiased results.

• Sample size, not population size, determines the precision of estimates.

• Be aware of and avoid common sources of bias in survey design and implementation.

• Always report sampling methods and potential limitations.