Sampling Distributions and Estimators: Essentials of Statistics Chapter 6 Study Notes

Sampling Distributions and Estimators

Introduction

Sampling distributions are fundamental in inferential statistics, allowing us to understand how sample statistics behave when repeatedly drawn from a population. This topic covers the behavior of sample proportions, means, and variances, and introduces the concepts of estimators, including unbiased and biased estimators.

General Behavior of Sampling Distributions

When samples of the same size are taken from the same population, certain properties emerge:

• Sample proportions and sample means tend to be normally distributed.

• The mean of all sample proportions (or sample means) equals the population proportion (or population mean).

Sampling Distributions for Proportions, Means, and Variances

Statistic	Sampling Procedure	Distribution Type
Proportion ($\hat{p}$)	Randomly select n values and find the proportion for each sample	Normal distribution
Mean ($\bar{x}$)	Randomly select n values and find the mean for each sample	Normal distribution
Variance ($s^2$)	Randomly select n values and find the variance for each sample	Skewed distribution (right-skewed)

Sampling Distribution of a Statistic

The sampling distribution of a statistic is the distribution of all values of the statistic when all possible samples of the same size n are taken from the same population.

Sampling Distribution of the Sample Proportion

The sampling distribution of the sample proportion ($\hat{p}$) is the distribution of sample proportions for all samples of size n from the population.

• Notation:

  • x: number of successes

  • n: sample size

  • N: population size

  • $\hat{p} = \frac{x}{n}$: sample proportion

  • $p = \frac{x}{N}$: population proportion

• Behavior:

  • The distribution of sample proportions tends to be normal.

  • The mean of all sample proportions equals the population proportion $p$.

Example: Rolling a die 5 times and finding the proportion of odd numbers (1, 3, 5). Repeating this process many times, the sample proportions are approximately normally distributed with a mean of 0.5 (since half the numbers on a die are odd).

Sampling Distribution of the Sample Mean

The sampling distribution of the sample mean ($\bar{x}$) is the distribution of all possible sample means for samples of size n from the population.

• The distribution of sample means tends to be normal.

• The mean of all sample means equals the population mean $\mu$.

Example: Rolling a die 5 times and finding the mean. Repeating this process many times, the sample means are approximately normally distributed with a mean of 3.5 (the mean of the numbers 1 through 6).

Sampling Distribution of the Sample Variance

The sampling distribution of the sample variance ($s^2$) is the distribution of all sample variances for samples of size n from the population.

• The distribution of sample variances is skewed to the right, not normal.

• The mean of all sample variances equals the population variance $\sigma^2$.

Example: Rolling a die 5 times and finding the variance. Repeating this process many times, the sample variances have a mean of 2.9 (the population variance for a fair die), and the distribution is right-skewed.

Estimators

An estimator is a statistic used to infer or estimate the value of a population parameter.

Unbiased Estimator

An unbiased estimator is a statistic whose sampling distribution has a mean equal to the corresponding population parameter.

• Unbiased estimators:

  • Sample proportion ($\hat{p}$)

  • Sample mean ($\bar{x}$)

  • Sample variance ($s^2$)

Biased Estimator

A biased estimator is a statistic whose sampling distribution does not have a mean equal to the population parameter.

• Biased estimators:

  • Median

  • Range

  • Sample standard deviation ($s$)

Important Note: The sample standard deviation $s$ does not target the population standard deviation $\sigma$, but the bias is small for large samples. Thus, $s$ is often used to estimate $\sigma$ even though it is technically a biased estimator.

Summary Table: Unbiased vs. Biased Estimators

Estimator	Unbiased?	Targets Parameter?
Sample Proportion ($\hat{p}$)	Yes	Population Proportion ($p$)
Sample Mean ($\bar{x}$)	Yes	Population Mean ($\mu$)
Sample Variance ($s^2$)	Yes	Population Variance ($\sigma^2$)
Sample Median	No	Does not target
Sample Range	No	Does not target
Sample Standard Deviation ($s$)	No (but often used)	Population Standard Deviation ($\sigma$)

Key Formulas

• Sample Proportion: $\hat{p} = \frac{x}{n}$

• Population Proportion: $p = \frac{x}{N}$

• Sample Mean: $\bar{x} = \frac{1}{n} \sum_{i=1}^{n} x_i$

• Sample Variance: $s^2 = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x})^2$

Applications and Importance

• Understanding sampling distributions is essential for making inferences about populations from samples.

• Unbiased estimators provide reliable estimates of population parameters, which is crucial for hypothesis testing and confidence intervals.

• Recognizing the difference between unbiased and biased estimators helps in selecting appropriate statistics for estimation.

Additional info: The examples provided use dice rolls to illustrate the behavior of sampling distributions, which is a common pedagogical approach in introductory statistics. The notes are based on slides from "Essentials of Statistics" by Mario F. Triola, Chapter 6, Section 6-3.