Sampling Distributions, Unbiasedness, and the Central Limit Theorem

The Concept of a Sampling Distribution

Parameter & Statistic

In statistics, it is important to distinguish between parameters and statistics, as they form the basis for inferential procedures.

• Parameter: A parameter is a numerical descriptive measure of a population. Because it is based on all the observations in the population, its value is almost always unknown in practice.

• Sample Statistic: A sample statistic is a numerical descriptive measure of a sample. It is calculated from the observations in the sample and is used to estimate the corresponding population parameter.

Population Parameters and Sample Statistics

The following table summarizes common population parameters and their corresponding sample statistics:

Population Parameter	Sample Statistic
Mean ($\mu$)	Sample Mean ($\bar{x}$)
Median ($\eta$)	Sample Median ($m$)
Variance ($\sigma^2$)	Sample Variance ($s^2$)
Standard Deviation ($\sigma$)	Sample Standard Deviation ($s$)
Binomial Proportion ($p$)	Sample Proportion ($\hat{p}$)

Sampling Distribution

The sampling distribution of a sample statistic calculated from a sample of n measurements is the probability distribution of that statistic. It describes how the statistic varies from sample to sample, assuming repeated random sampling from the same population.

Example: Coin-Tossing Experiment

Suppose we toss a fair coin twice and let x be the number of heads observed. The possible outcomes and their probabilities are:

Outcome (Toss 1, Toss 2)	Probability
(x = 1, x = 1)	1/4
(x = 1, x = 0)	1/4
(x = 0, x = 1)	1/4
(x = 0, x = 0)	1/4

The sample mean $\bar{x}$ for each outcome is:

$\bar{x}$	Probability $p(\bar{x})$
0	1/4
0.5	1/2
1	1/4

This table represents the sampling distribution of the sample mean for two coin tosses.

Properties of Sampling Distributions: Unbiasedness and Minimum Variance

Point Estimator

A point estimator of a population parameter is a rule or formula that tells us how to use the sample data to calculate a single number that can be used as an estimate of the population parameter.

Unbiased and Biased Estimates

• Unbiased Estimator: If the mean of the sampling distribution of a sample statistic equals the population parameter it is intended to estimate, the statistic is said to be an unbiased estimator of the parameter.

• Biased Estimator: If the mean of the sampling distribution is not equal to the parameter, the statistic is said to be a biased estimator of the parameter.

For example, the sample mean $\bar{x}$ is an unbiased estimator of the population mean $\mu$.

The Sampling Distribution of the Sample Mean and the Central Limit Theorem

Properties of the Sampling Distribution of the Mean

• The mean of the sampling distribution of the sample mean equals the mean of the sampled population:

$E(\bar{x}) = \mu$

• The standard deviation of the sampling distribution of the sample mean (also called the standard error) is:

$\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}$

where $\sigma$ is the population standard deviation and $n$ is the sample size.

Theorem 5.1: Normality of the Sampling Distribution

• If a random sample of $n$ observations is selected from a population with a normal distribution, the sampling distribution of $\bar{x}$ will also be normal.

Central Limit Theorem (CLT)

The Central Limit Theorem states that for a random sample of $n$ observations drawn from any population with mean $\mu$ and standard deviation $\sigma$, when $n$ is sufficiently large, the sampling distribution of $\bar{x}$ will be approximately normal with mean $\mu$ and standard deviation $\sigma/\sqrt{n}$, regardless of the shape of the population distribution.

• The larger the sample size, the better the normal approximation.

• For most populations, $n \geq 30$ is considered sufficient for the normal approximation to be reasonable.

Example: Using the Central Limit Theorem

Suppose a random sample of $n = 36$ is taken from a population with mean $\mu = 80$ and standard deviation $\sigma = 6$. The population is extremely skewed. The sampling distribution of $\bar{x}$ will have:

• Mean: $\mu_{\bar{x}} = 80$

• Standard Error: $\sigma_{\bar{x}} = \frac{6}{\sqrt{36}} = 1$

To find the probability that $\bar{x} > 82$:

• Compute the z-score:

$z = \frac{82 - 80}{1} = 2$

• From standard normal tables, $P(z > 2) = 0.0228$

The Sampling Distribution of the Sample Proportion

Sample Proportion

The sample proportion $\hat{p}$ is a good estimator of the population proportion $p$. The quality of this estimator depends on the properties of its sampling distribution.

• The mean of the sampling distribution of $\hat{p}$ is $E(\hat{p}) = p$ (unbiased estimator).

• The standard deviation (standard error) of the sampling distribution is:

$\sigma_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}}$

• For large samples, the sampling distribution of $\hat{p}$ is approximately normal if $np \geq 15$ and $n(1-p) \geq 15$.

Summary Table: Properties of Sample Mean and Proportion

Statistic	Mean of Sampling Distribution	Standard Error	Approximate Distribution (Large n)
Sample Mean ($\bar{x}$)	$\mu$	$\frac{\sigma}{\sqrt{n}}$	Normal (by CLT)
Sample Proportion ($\hat{p}$)	$p$	$\sqrt{\frac{p(1-p)}{n}}$	Normal (if $np \geq 15$, $n(1-p) \geq 15$)