Estimating Parameters and Constructing Confidence Intervals in Statistics

Estimating the Value of a Parameter

Introduction

This chapter focuses on statistical methods for estimating unknown population parameters, such as proportions, means, and standard deviations, using sample data. It covers point estimation, confidence intervals, determining sample sizes, and introduces both parametric and nonparametric (bootstrap) approaches.

Estimating a Population Proportion

Point Estimate for the Population Proportion

• Point Estimate: The value of a statistic that estimates the value of a parameter. For a population proportion, the point estimate is the sample proportion, $\hat{p}$.

• Formula: $\hat{p} = \frac{x}{n}$, where $x$ is the number of individuals in the sample with a specified characteristic and $n$ is the sample size.

• Example: In a poll of 1015 Americans, 458 said their federal income tax is too high. $\hat{p} = \frac{458}{1015} = 0.451$.

Confidence Interval for the Population Proportion

• Confidence Interval: An interval of numbers based on a point estimate, expressing the range within which the population parameter is expected to lie with a certain level of confidence.

• Level of Confidence: The expected proportion of intervals that will contain the parameter if many samples are taken. Common levels are 90%, 95%, and 99%.

• Formula: For a $(1-\alpha)\times100\%$ confidence interval: $\hat{p} \pm z_{\alpha/2} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$ where $z_{\alpha/2}$ is the critical value from the standard normal distribution.

• Margin of Error (E): $E = z_{\alpha/2} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$

• Interpretation: A 95% confidence interval means that if 100 different samples are taken, about 95 of the intervals will contain the true population proportion.

• Caution: The confidence level does not represent the probability that the specific interval contains the parameter; the parameter is fixed.

• Example: For $\hat{p} = 0.451$, margin of error $E = 0.04$, the 95% confidence interval is $(0.411, 0.491)$.

Determining Sample Size for Estimating a Proportion

• Formula (with prior estimate $\hat{p}$): $n = \frac{z_{\alpha/2}^2 \hat{p}(1-\hat{p})}{E^2}$

• Formula (no prior estimate): $n = \frac{z_{\alpha/2}^2 \cdot 0.25}{E^2}$

• Example: To estimate a proportion within 2 percentage points (E = 0.02) at 90% confidence, with $\hat{p} = 0.10$, $n = 609$. Without prior estimate, $n = 1692$.

Estimating a Population Mean

Point Estimate for the Population Mean

• Sample Mean ($\bar{x}$): The point estimate for the population mean $\mu$.

• Example: For 16 Toyota Camry owners, $\bar{x} = 28.1$ mpg.

Student’s t-Distribution

• Definition: When sampling from a normal population and the population standard deviation is unknown, the distribution of $t = \frac{\bar{x} - \mu}{s/\sqrt{n}}$ follows Student’s t-distribution with $n-1$ degrees of freedom.

• Properties:

  • Symmetric, bell-shaped, centered at 0.

  • More spread (longer tails) than the standard normal distribution, especially for small $n$.

  • As $n$ increases, the t-distribution approaches the standard normal distribution.

Constructing a Confidence Interval for the Population Mean

• Formula: $\bar{x} \pm t_{\alpha/2,\,df} \frac{s}{\sqrt{n}}$ where $t_{\alpha/2,\,df}$ is the critical value from the t-distribution with $df = n-1$.

• Normality Condition: For small samples ($n < 30$), check for normality and outliers using plots. For larger samples, the Central Limit Theorem applies.

• Example: For $n = 16$, $\bar{x} = 28.1$, $s = 2.38$, 95% confidence interval is $(26.83, 29.37)$ mpg.

Determining Sample Size for Estimating a Mean

• Formula: $n = \left( \frac{z_{\alpha/2} s}{E} \right)^2$ (rounded up)

• Example: To estimate mean mpg within 0.5 mpg at 95% confidence, $n = 88$.

Estimating a Population Standard Deviation

Chi-Square Distribution

• Definition: If a sample of size $n$ is taken from a normal population, $\chi^2 = \frac{(n-1)s^2}{\sigma^2}$ follows a chi-square distribution with $n-1$ degrees of freedom.

• Properties: Not symmetric; shape depends on degrees of freedom; values are nonnegative.

Confidence Interval for Population Variance and Standard Deviation

• Formula for Variance: Lower bound: $\frac{(n-1)s^2}{\chi^2_{\alpha/2,\,df}}$ Upper bound: $\frac{(n-1)s^2}{\chi^2_{1-\alpha/2,\,df}}$

• For Standard Deviation: Take the square root of the bounds for variance.

• Example: For 12 Corvettes, 90% confidence interval for standard deviation is ($1955, 4055$) dollars.

Bootstrapping

Bootstrap Method for Estimating Parameters

• Definition: Bootstrapping is a computer-intensive, nonparametric method that estimates parameters by resampling with replacement from the observed data.

• Requirements: The bootstrap distribution should be centered near the original sample statistic and be symmetric.

• Algorithm:

  1. Draw $B$ bootstrap samples (with replacement) of size $n$ from the original data.

  2. Compute the statistic (e.g., mean) for each sample.

  3. Use the distribution of these statistics to estimate confidence intervals (e.g., 2.5th and 97.5th percentiles for a 95% interval).

• Example: For 16 Camry mpg values, the 95% bootstrap confidence interval for the mean is $(27.05, 29.22)$ mpg.

Bootstrapping for Proportions

• Encode data as 0 (failure) and 1 (success); the sample proportion is the mean of these values.

Considerations When Using Bootstrapping

• Small samples may not accurately reflect the true sampling distribution.

• Bootstrap standard errors may be slightly underestimated.

Summary Table: Confidence Interval Methods

Parameter	Point Estimate	Confidence Interval Formula	Distribution
Proportion ($p$)	$\hat{p}$	$\hat{p} \pm z_{\alpha/2} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$	Normal (z)
Mean ($\mu$)	$\bar{x}$	$\bar{x} \pm t_{\alpha/2,\,df} \frac{s}{\sqrt{n}}$	t-distribution
Variance ($\sigma^2$)	$s^2$	$\left( \frac{(n-1)s^2}{\chi^2_{\alpha/2,\,df}}, \frac{(n-1)s^2}{\chi^2_{1-\alpha/2,\,df}} \right)$	Chi-square

Appendix: Key Definitions

• Point Estimate: A single value used to estimate a population parameter.

• Confidence Interval: A range of values, derived from sample statistics, that is likely to contain the population parameter.

• Margin of Error: The maximum expected difference between the point estimate and the true parameter.

• Critical Value: The number of standard errors to move away from the point estimate to achieve the desired confidence level.

• Bootstrap: A resampling method for estimating the distribution of a statistic by sampling with replacement from the data.