Sampling Distributions and the Central Limit Theorem: Study Notes

Sampling Distributions and Estimators

Introduction to Sampling Distributions

Sampling distributions are foundational in inferential statistics, allowing us to understand how sample statistics behave when drawn from a population. Instead of focusing on individual data values, we analyze the distribution of statistics such as sample means or proportions.

• Sampling Distribution of a Statistic: The probability distribution of a statistic (e.g., sample mean, sample proportion) when all possible samples of a given size are drawn from the same population.

• Typically represented as a probability histogram, formula, or table.

Sampling Distribution of the Sample Proportion

The sampling distribution of the sample proportion describes the distribution of sample proportions (denoted as p̂) from all possible samples of the same size from a population.

• Notation:

  • x = number of successes

  • n = sample size

  • N = population size

  • p̂ = x/n (sample proportion)

  • p = x/N (population proportion)

• Behavior:

  • The distribution of sample proportions tends to approximate a normal distribution as sample size increases.

  • The mean of all sample proportions (p̂) equals the population proportion (p).

Example: Rolling a die 5 times and finding the proportion of odd numbers. Repeating this process many times, the distribution of sample proportions is approximately normal with mean 0.5 (since half the numbers on a die are odd).

Sampling Distribution of the Sample Mean

The sampling distribution of the sample mean (x̄) is the distribution of all possible sample means from samples of the same size drawn from a population.

• Behavior:

  • The distribution of sample means tends to be normal, especially as sample size increases (Central Limit Theorem).

  • The mean of the sample means equals the population mean (μ).

Example: Rolling a die 5 times and finding the mean. Repeating this process many times, the distribution of sample means is approximately normal with mean 3.5 (the population mean for a fair die).

Estimators: Unbiased and Biased

An estimator is a statistic used to infer the value of a population parameter.

• Unbiased Estimator: A statistic whose sampling distribution has a mean equal to the population parameter it estimates.

• Biased Estimator: A statistic whose sampling distribution does not target the population parameter.

Unbiased Estimators	Biased Estimators
Sample Proportion (p̂)	Median
Sample Mean (x̄)	Range
Sample Variance (s²)	Sample Standard Deviation (s)

Note: The sample standard deviation (s) is a biased estimator of the population standard deviation (σ), but the bias is small for large samples, so s is often used in practice.

Sampling with Replacement

• Sampling with replacement ensures independent events, simplifying probability calculations.

• For large populations and small samples, the difference between sampling with and without replacement is negligible.

The Central Limit Theorem (CLT)

Statement of the Central Limit Theorem

The Central Limit Theorem is a key result in statistics, stating that for sufficiently large sample sizes (n > 30), the sampling distribution of the sample mean (x̄) will be approximately normal, regardless of the population's distribution.

• Mean of sampling distribution: $\mu_{\bar{x}} = \mu$

• Standard deviation (standard error): $\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}$

Practical Rules for Applications

• If the population is normal or n > 30, the sampling distribution of x̄ is approximately normal.

• If the population is not normal and n ≤ 30, the sampling distribution of x̄ may not be normal, and normal-based methods do not apply.

• Z-score for sample mean:

  • $z = \frac{\bar{x} - \mu}{\sigma/\sqrt{n}}$

Considerations for Problem Solving

• Check Requirements: Confirm that the population is normal or n > 30 before applying normal distribution methods to sample means.

• Individual Value vs. Sample Mean:

  • For an individual value: $z = \frac{x - \mu}{\sigma}$

  • For a sample mean: $z = \frac{\bar{x} - \mu}{\sigma/\sqrt{n}}$

Summary Table: Key Formulas

Statistic	Mean of Sampling Distribution	Standard Deviation (Standard Error)
Sample Proportion (p̂)	p	$\sqrt{\frac{p(1-p)}{n}}$
Sample Mean (x̄)	μ	$\frac{\sigma}{\sqrt{n}}$

Additional info: The standard error is sometimes denoted as SEM (Standard Error of the Mean).