Sampling Distributions and the Central Limit Theorem

Sampling Distributions

The Concept of a Sampling Distribution

In statistics, a sampling distribution is the probability distribution of a given statistic based on a random sample. It is fundamental to inferential statistics, as it allows us to understand the variability of sample statistics and make probabilistic statements about population parameters.

• Parameter: A numerical descriptive measure of a population (e.g., population mean μ, population standard deviation σ).

• Statistic: A numerical descriptive measure of a sample (e.g., sample mean \bar{x}, sample standard deviation s).

• Sampling Distribution: The probability distribution of a sample statistic calculated from all possible samples of a fixed size n from a population.

Example: If a population consists of the values 1, 2, 3, and 4, and we take all possible samples of size 2 (with replacement), the distribution of the sample means forms the sampling distribution of the mean.

Common Statistics and Parameters

Sample Statistic	Population Parameter
Sample Mean (\bar{x})	Population Mean (μ)
Sample Variance (s^2)	Population Variance (σ^2)
Sample Standard Deviation (s)	Population Standard Deviation (σ)
Sample Proportion (\hat{p})	Population Proportion (p)

Properties of Sampling Distributions

Unbiasedness and Minimum Variance

A point estimator is a rule or formula that tells us how to use sample data to estimate a population parameter. Two important properties of estimators are:

• Unbiasedness: An estimator is unbiased if the mean of its sampling distribution equals the true value of the parameter it estimates. For example, the sample mean \bar{x} is an unbiased estimator of the population mean μ.

• Minimum Variance: Among all unbiased estimators, the one with the smallest variance is preferred. This is called the minimum-variance unbiased estimator (MVUE).

Example: If the expected value of the sample mean equals the population mean, then \bar{x} is an unbiased estimator of μ.

The Sampling Distribution of the Sample Mean and the Central Limit Theorem

Properties of the Sampling Distribution of the Mean

The sampling distribution of the sample mean \bar{x} has the following properties:

• Mean: $\mu_{\bar{x}} = E(\bar{x}) = \mu$

• Standard Deviation (Standard Error): $\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}$

The standard deviation of the sampling distribution is called the standard error of the mean.

The Central Limit Theorem (CLT)

The Central Limit Theorem states that, for a sufficiently large sample size (typically n ≥ 30), the sampling distribution of the sample mean \bar{x} will be approximately normal, regardless of the shape of the population distribution. This is a cornerstone of inferential statistics.

• Mean: $\mu_{\bar{x}} = \mu$

• Standard Error: $\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}$

• Normality: As n increases, the distribution of \bar{x} approaches normality.

Standardizing the Sample Mean: To find probabilities, we standardize using the z-score:

• $z = \frac{\bar{x} - \mu}{\sigma / \sqrt{n}}$

Example: If the mean call duration is 8 minutes (σ = 2), and we take samples of 25 calls, the standard error is $\sigma_{\bar{x}} = \frac{2}{\sqrt{25}} = 0.4$. The probability that the sample mean is between 7.8 and 8.2 minutes can be found using the standard normal distribution.

The Sampling Distribution of the Sample Proportion

Sample Proportion

The sample proportion (\hat{p}) is used to estimate the population proportion (p). The sampling distribution of \hat{p} has properties similar to those of the sample mean:

• Mean: $\mu_{\hat{p}} = p$

• Standard Error: $\sigma_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}}$

• For large n, the sampling distribution of \hat{p} is approximately normal (by the Central Limit Theorem).

Example: If 60% of a population prefers a product (p = 0.6), and a sample of 100 is taken, the standard error is $\sqrt{\frac{0.6 \times 0.4}{100}} = 0.049$.

Key Ideas and Summary Table

• Sampling distribution: Theoretical probability distribution of a statistic in repeated sampling.

• Unbiased estimator: Statistic whose mean equals the parameter it estimates.

• Central Limit Theorem: For large n, the sampling distribution of the sample mean or proportion is approximately normal.

• MVUE: Minimum-variance unbiased estimator.

Statistic	Mean of Sampling Distribution	Standard Error	Distribution (Large n)
Sample Mean (\bar{x})	μ	σ/√n	Normal
Sample Proportion (\hat{p})	p	√[p(1-p)/n]	Normal