Statistical Inference: Sampling Distributions, Estimation, and Hypothesis Testing

Sampling Distributions

Sampling Distribution of the Sample Mean

The sampling distribution of the sample mean describes the distribution of sample means over repeated sampling from the same population. It is foundational for making inferences about population parameters.

• Shape: If the population is normal, the sampling distribution of the mean is also normal. For large samples (n > 30), the Central Limit Theorem ensures approximate normality regardless of the population shape.

• Mean: The mean of the sampling distribution is equal to the population mean, $\mu$.

• Standard Deviation (Standard Error): The standard deviation of the sampling distribution is $\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}$, where $\sigma$ is the population standard deviation and $n$ is the sample size.

Example: If upper leg length in 20-29 year old males is normal with mean 43.7 cm and standard deviation 4.2 cm, for a sample of 12 males:

• Shape: Normal

• Mean: 43.7 cm

• Standard Error: $\frac{4.2}{\sqrt{12}} \approx 1.21$ cm

Sampling Distribution of Proportions

The sampling distribution of the sample proportion describes the distribution of sample proportions over repeated samples from the same population.

• Shape: Approximately normal if $np \geq 10$ and $n(1-p) \geq 10$.

• Mean: $p$ (population proportion)

• Standard Deviation (Standard Error): $\sigma_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}}$

Example: If 15% of Americans have hearing trouble and a sample of 120 is taken:

• Shape: Approximately normal

• Mean: 0.15

• Standard Error: $\sqrt{\frac{0.15 \times 0.85}{120}} \approx 0.032$

Estimation of Population Parameters

Point Estimation and Confidence Intervals for Means

Point estimation provides a single value as an estimate of a population parameter. Confidence intervals give a range of plausible values for the parameter.

• Point Estimate: The sample mean $\bar{x}$ is the best point estimate for the population mean $\mu$.

• Confidence Interval for Mean (when $\sigma$ unknown):

$\bar{x} \pm t^* \left( \frac{s}{\sqrt{n}} \right)$

• $t^*$ is the critical value from the t-distribution with $n-1$ degrees of freedom.

• $s$ is the sample standard deviation.

Margin of Error: $t^* \left( \frac{s}{\sqrt{n}} \right)$

Example: A sample of 92 yields $\bar{x} = 356.1$, $s = 185.7$. For a 95% confidence interval, find $t^*$ for $df = 91$ and calculate the interval.

Properties of the t-Distribution

• The t-distribution is symmetric and centered at 0.

• It has heavier tails than the normal distribution, especially for small sample sizes.

• As degrees of freedom increase, the t-distribution approaches the normal distribution.

• Degrees of freedom: $df = n - 1$

Additional info: The image above shows how the t-distribution changes with different degrees of freedom. As the sample size increases, the t-distribution becomes more similar to the standard normal distribution.

Confidence Intervals for Proportions

• Point Estimate: $\hat{p}$ (sample proportion)

• Confidence Interval: $\hat{p} \pm z^* \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$

• $z^*$ is the critical value from the standard normal distribution for the desired confidence level.

Hypothesis Testing

Hypothesis Testing for Means

Hypothesis testing is used to assess claims about population means.

• Null Hypothesis ($H_0$): The default assumption (e.g., $\mu = \mu_0$).

• Alternative Hypothesis ($H_a$): The claim to be tested (e.g., $\mu \neq \mu_0$, $\mu > \mu_0$, or $\mu < \mu_0$).

• Test Statistic: $t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}$

• P-value: The probability of observing a test statistic as extreme as, or more extreme than, the observed value under $H_0$.

• Significance Level ($\alpha$): Common values are 0.01, 0.05, 0.10.

• Decision Rule: If $p$-value < $\alpha$, reject $H_0$; otherwise, fail to reject $H_0$.

Types of Errors

• Type I Error: Rejecting $H_0$ when it is true (false positive). Probability = $\alpha$.

• Type II Error: Failing to reject $H_0$ when $H_a$ is true (false negative). Probability = $\beta$.

Hypothesis Testing for Proportions

• Test Statistic: $z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}$

• P-value: Calculated based on the direction of the alternative hypothesis.

Two-Sample Inference

Two-Sample t-Test for Means

Used to compare the means of two independent groups.

• Test Statistic: $t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$

• Confidence Interval for Difference: $(\bar{x}_1 - \bar{x}_2) \pm t^* \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$

Two-Sample z-Test for Proportions

Used to compare the proportions of two independent groups.

• Test Statistic: $z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$

• $\hat{p}$ is the pooled proportion: $\hat{p} = \frac{x_1 + x_2}{n_1 + n_2}$

Summary Table: Key Formulas

Parameter	Point Estimate	Standard Error	Confidence Interval
Mean ($\mu$)	$\bar{x}$	$\frac{s}{\sqrt{n}}$	$\bar{x} \pm t^* \frac{s}{\sqrt{n}}$
Proportion ($p$)	$\hat{p}$	$\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$	$\hat{p} \pm z^* \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$
Difference of Means ($\mu_1 - \mu_2$)	$\bar{x}_1 - \bar{x}_2$	$\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$	$(\bar{x}_1 - \bar{x}_2) \pm t^* \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$
Difference of Proportions ($p_1 - p_2$)	$\hat{p}_1 - \hat{p}_2$	$\sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}$	$(\hat{p}_1 - \hat{p}_2) \pm z^* \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}$