Sampling Distributions and the Central Limit Theorem

Sampling Distributions

Definition and Overview

The sampling distribution of a statistic (such as a sample proportion, sample mean, or sample variance) is the probability distribution of all possible values of that statistic when all possible samples of a fixed size n are drawn from the same population. Sampling distributions are fundamental in inferential statistics, as they describe the variability of sample statistics and allow for estimation and hypothesis testing.

• Statistic: A numerical measure calculated from a sample (e.g., sample mean, sample proportion).

• Sampling Distribution: The distribution of a statistic over all possible samples of size n from a population.

• Representation: Sampling distributions are often visualized as probability histograms, formulas, or tables.

Sample Proportions

Definition and Properties

The sample proportion is the fraction of items in a sample that possess a certain characteristic. The sampling distribution of sample proportions describes how these proportions vary from sample to sample.

• Notation: $\hat{p}$ denotes the sample proportion; $p$ denotes the population proportion.

• Distribution: For large sample sizes, the sampling distribution of sample proportions tends to be normal (by the Central Limit Theorem).

• Mean: $\mu_{\hat{p}} = p$

• Standard Deviation: $\sigma_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}}$

Example: If 40% of a population supports a policy ($p = 0.4$), the sample proportion $\hat{p}$ from random samples of size $n$ will vary, but the distribution of $\hat{p}$ will be centered at 0.4.

Sample Means

Definition and Properties

The sample mean is the average value of a variable in a sample. The sampling distribution of sample means describes how these averages vary across samples.

• Notation: $\bar{x}$ denotes the sample mean; $\mu$ denotes the population mean.

• Distribution: The sampling distribution of sample means tends to be normal, especially as sample size increases (Central Limit Theorem).

• Mean: $\mu_{\bar{x}} = \mu$

• Standard Deviation: $\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}$

Example: If the population mean height is 170 cm, the sample means from random samples of size $n$ will vary, but their distribution will be centered at 170 cm.

Sample Variances

Definition and Properties

The sample variance measures the spread of values in a sample. The sampling distribution of sample variances describes how these variances vary across samples.

• Notation: $s^2$ denotes the sample variance; $\sigma^2$ denotes the population variance.

• Distribution: The sampling distribution of sample variances is skewed to the right and does not follow a normal distribution.

• Mean: The mean of the sample variances equals the population variance: $E[s^2] = \sigma^2$

• Implication: Because the distribution is not normal, standard methods for normal distributions do not apply.

Example: If the population variance is 25, the sample variances from random samples will vary, but their average will be close to 25, though the distribution is skewed.

The Central Limit Theorem (CLT)

Statement and Importance

The Central Limit Theorem is a fundamental result in statistics. It states that, for a sufficiently large sample size, the sampling distribution of the sample mean (or sample proportion) will be approximately normal, regardless of the shape of the population distribution.

• Formal Statement: If $X_1, X_2, ..., X_n$ are independent, identically distributed random variables with mean $\mu$ and variance $\sigma^2$, then the distribution of $\bar{X}$ approaches normal as $n$ increases.

• Mathematical Expression: $\bar{X} \sim N\left(\mu, \frac{\sigma^2}{n}\right)$ for large $n$

• Applications: Allows for confidence intervals and hypothesis tests using normal distribution methods.

Example: Even if the population is skewed, the distribution of sample means from samples of size $n \geq 30$ will be nearly normal.

Summary Table: Sampling Distributions

The following table summarizes the behavior of sampling distributions for proportions, means, and variances:

Statistic	Sampling Distribution Shape	Targets Population Value?	Notes
Sample Proportion ($\hat{p}$)	Normal (for large $n$)	Yes ($p$)	CLT applies; used for binomial proportions
Sample Mean ($\bar{x}$)	Normal (for large $n$)	Yes ($\mu$)	CLT applies; used for quantitative data
Sample Variance ($s^2$)	Skewed right	Yes ($\sigma^2$)	Not normal; special methods required