6.3

Sampling Distributions and Estimators

Introduction

Sampling distributions are fundamental in inferential statistics, allowing us to understand how sample statistics behave in repeated sampling from a population. This section explores the behavior of sample proportions, means, and variances, and introduces the concepts of estimators, including unbiased and biased estimators.

General Behavior of Sampling Distributions

• Sampling Distribution: The probability distribution of a given statistic based on a random sample.

• When samples of the same size are repeatedly drawn from the same population, the following properties are observed:

  • Sample proportions and sample means tend to be normally distributed.

  • The mean of all sample proportions (or means) equals the population proportion (or mean).

Table: Summary of Sampling Distributions

Statistic	Sampling Procedure	Distribution Shape	Population Parameter Targeted
Proportion ($\hat{p}$)	Randomly select $n$ values and find the proportion for each sample	Normal	Population Proportion ($p$)
Mean ($\bar{x}$)	Randomly select $n$ values and find the mean for each sample	Normal	Population Mean ($\mu$)
Variance ($s^2$)	Randomly select $n$ values and find the variance for each sample	Skewed Right	Population Variance ($\sigma^2$)

Sampling Distribution of a Statistic

• The sampling distribution of a statistic is the distribution of all possible values of that statistic when all possible samples of the same size $n$ are taken from the same population.

Sampling Distribution of the Sample Proportion

• The sampling distribution of the sample proportion is the distribution of sample proportions ($\hat{p}$) from all samples of size $n$ drawn from the population.

Notations for Proportions

• $x$ = number of successes

• $n$ = sample size

• $N$ = population size

• $\hat{p} = x/n$ denotes the sample proportion

• $p = x/N$ denotes the population proportion

• Note: Symbols with a hat (e.g., $\hat{p}$) or bar (e.g., $\bar{x}$) represent statistics, not parameters.

Behavior of Sample Proportions

• The distribution of sample proportions tends to approximate a normal distribution as sample size increases.

• Sample proportions target the population proportion: the mean of all sample proportions equals the population proportion ($E[\hat{p}] = p$).

Example: Sampling Distributions of the Sample Proportion

• Scenario: Roll a die 5 times and find the proportion of odd numbers (1, 3, or 5). Repeat this process many times.

• Observation: The distribution of sample proportions from many repetitions is approximately normal, centered at the population proportion (0.5 for odd numbers).

Sampling Distribution of the Sample Mean

• The sampling distribution of the sample mean is the distribution of all possible sample means ($\bar{x}$) from all samples of size $n$ drawn from the population.

Behavior of Sample Means

• The distribution of sample means tends to be normal (especially as $n$ increases, by the Central Limit Theorem).

• Sample means target the population mean: $E[\bar{x}] = \mu$.

Example: Sampling Distribution of the Sample Mean

• Scenario: Roll a die 5 times, find the mean of the results, and repeat this process many times.

• Observation: The distribution of sample means is approximately normal, centered at the population mean ($\mu = 3.5$ for a fair die).

Sampling Distribution of the Sample Variance

• The sampling distribution of the sample variance is the distribution of all possible sample variances ($s^2$) from all samples of size $n$ drawn from the population.

Behavior of Sample Variances

• The distribution of sample variances is typically skewed to the right, not normal.

• Sample variances target the population variance: $E[s^2] = \sigma^2$.

Example: Sampling Distributions of the Sample Variance

• Scenario: Roll a die 5 times, find the variance of the results, and repeat this process many times.

• Observation: The distribution of sample variances is skewed right, with a mean equal to the population variance ($\sigma^2 = 2.9$ for a fair die).

Estimators

• An estimator is a statistic used to infer (estimate) the value of a population parameter.

Unbiased Estimator

• An unbiased estimator is a statistic whose sampling distribution has a mean equal to the corresponding population parameter.

• Examples of unbiased estimators:

  • Sample proportion ($\hat{p}$) for population proportion ($p$)

  • Sample mean ($\bar{x}$) for population mean ($\mu$)

  • Sample variance ($s^2$) for population variance ($\sigma^2$)

Biased Estimator

• A biased estimator is a statistic whose sampling distribution does not have a mean equal to the corresponding population parameter.

• Examples of biased estimators:

  • Sample median for population median

  • Sample range for population range

  • Sample standard deviation ($s$) for population standard deviation ($\sigma$)

• Note: The bias of $s$ as an estimator for $\sigma$ is relatively small for large samples, so $s$ is often used to estimate $\sigma$ in practice.

Summary Table: Unbiased vs. Biased Estimators

Statistic	Parameter Estimated	Unbiased?
Sample Proportion ($\hat{p}$)	Population Proportion ($p$)	Yes
Sample Mean ($\bar{x}$)	Population Mean ($\mu$)	Yes
Sample Variance ($s^2$)	Population Variance ($\sigma^2$)	Yes
Sample Median	Population Median	No
Sample Range	Population Range	No
Sample Standard Deviation ($s$)	Population Standard Deviation ($\sigma$)	No (but bias is small for large $n$)

Key Formulas

• Sample Proportion: $\hat{p} = \frac{x}{n}$

• Population Proportion: $p = \frac{x}{N}$

• Sample Mean: $\bar{x} = \frac{1}{n} \sum_{i=1}^n x_i$

• Sample Variance: $s^2 = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})^2$

Additional info: The Central Limit Theorem (CLT) underpins the normality of sample means and proportions for large sample sizes, even when the population distribution is not normal. This is foundational for inferential statistics.