Statistical Inference for Regression and One-Way ANOVA: Estimation, Confidence Intervals, and Model Comparison

Statistical Inference for Regression

Estimation and Sources of Variation

Statistical inference in regression involves estimating model parameters and understanding the sources of variation in predictions. Estimation allows us to quantify uncertainty about parameters such as the slope and intercept, and to make predictions about mean and individual responses.

• Estimation refers to the process of using sample data to infer the value of population parameters.

• Sources of error include sampling variability, model assumptions, and measurement error.

• Confidence intervals (CI) provide a range of plausible values for an estimated parameter, reflecting uncertainty due to sampling.

General Form of Confidence Intervals

The general formula for a confidence interval is:

• Best bet ± Critical value × SE(Best bet)

Specific cases include:

• Proportion: $\hat{p} \pm z^* \cdot SE(\hat{p})$

• Mean: $\bar{y} \pm t^* \cdot SE(\bar{y})$

• Regression intercept: $b_0 \pm t^* \cdot SE(b_0)$

• Regression slope coefficient: $b_1 \pm t^* \cdot SE(b_1)$

Confidence Intervals for Mean

When estimating the mean, the confidence interval depends on whether the population standard deviation ($\sigma$) is known or unknown:

• If $\sigma$ is known: $\bar{y} \pm z^* \frac{\sigma}{\sqrt{n}}$

• If $\sigma$ is unknown: $\bar{y} \pm t^* \frac{s}{\sqrt{n}}$

Where $z^*$ and $t^*$ are critical values from the standard normal and t-distributions, respectively.

Types of Confidence Intervals in Regression

In regression, two main types of confidence intervals are used:

• CI for Mean Response (line): The interval for the mean value of $y$ at a given $x$.

• CI for Individual Response (point): The interval for a single predicted value of $y$ at a given $x$.

CI for mean response is narrower than CI for individual response because the latter includes both the uncertainty in the mean and the residual variation.

Prediction in Regression

Regression models can be used to predict:

• Mean response: $\hat{y} \pm ME$

• Individual response: $\hat{y} \pm t^* \cdot SE(\hat{y})$

The uncertainty about the mean decreases with increasing sample size, while the residual variation around the line does not depend on sample size.

Error Sources in Regression

There are two main sources of uncertainty in regression:

• Uncertainty about the line of best fit: Increases with distance from the mean of $x$.

• Residual variation around the line: Remains constant regardless of sample size.

Model Comparison and Assumptions

Comparing regression models involves checking assumptions and evaluating fit:

• Model 1: Original data

• Model 2: Original data with outlier removed

• Model 3: Log-log transformed data

Best model selection criteria:

• Meets statistical assumptions

• Fits the data well (high $R^2$)

• Is parsimonious (as simple as possible)

• Makes sense biologically

Checking Residual Plots

Residual plots are used to assess model assumptions:

• Linearity of relationship

• Normality of residuals

• Constant variance (homoscedasticity)

• Influential points

Transformations (e.g., log-log) can help address violations of assumptions.

Reporting Regression Results

Regression results should include:

• Methods: Description of model, transformations, and variables.

• Results: Estimated coefficients, $R^2$, significance levels, and residual analysis.

Example (log-log model):

• $\log(\text{age}) = 2.240 + 0.725 \cdot \log(\text{proportion.black}) + \text{error}$

• Back-transform to original scale: $\text{age} = \exp(\log(\text{age}))$

One-Way ANOVA

Introduction to ANOVA

One-way Analysis of Variance (ANOVA) is used to compare means across multiple groups to determine if at least one group mean is significantly different from the others.

• Boxplots, normal probability plots, and variance comparisons are used to check assumptions.

• ANOVA partitions total variation into model and residual sums of squares.

Key Concepts and Calculations

• Total sum of squares (SST): Measures total variation in the data.

• Model sum of squares (SSM): Variation explained by group differences.

• Residual sum of squares (SSR): Variation within groups.

• Degrees of freedom (df): Number of independent values in each sum of squares.

• Mean sum of squares: Sum of squares divided by corresponding degrees of freedom.

Interpreting ANOVA Output

ANOVA output includes F-statistics and p-values to test the null hypothesis that all group means are equal.

• If the F-test is significant, at least one group mean differs.

• Pairwise comparisons can be performed to identify which means differ.

Recommended Problems (from textbook)

Edition	Problem	Topic
3rd & 4th	24.13	Identify hypotheses, df, basic interpretation
3rd & 4th	24.21	Pairwise comparisons of means
3rd & 4th	24.25	Check ANOVA assumptions
3rd & 4th	24.27	Interpret ANOVA output and figures
2nd	25.3	Identify hypotheses, df, basic interpretation
2nd	25.7	Interpret ANOVA output and figures
2nd	25.11	Check ANOVA assumptions
2nd	25.21	Pairwise comparisons of means

Application Example: Lion Age Prediction

Regression Model for Lion Age

Regression is used to predict lion age based on black nose pigmentation. Three models are compared:

• Model 1: $\hat{\text{age}} = 0.879 + 10.647 \cdot \text{proportion.black}$

• Model 2: $\hat{\text{age}} = 1.294 + 8.85 \cdot \text{proportion.black}$ (outlier removed)

• Model 3: $\log(\text{age}) = 2.240 + 0.725 \cdot \log(\text{proportion.black})$ (log-log transformed)

Back-transform for Model 3: $\text{age} = \exp(1.87) = 6.5$ years

Confidence Intervals in Practice

To determine if a lion is older than 6 years based on pigmentation, use the appropriate confidence interval:

• 95% CI for mean response: for estimating average age

• 95% CI for individual response: for estimating age of a specific lion

Review Problem: Brain Mass and Glia-Neuron Ratio

Type	Brain mass (g)	Glia-neuron ratio
Human	1357	1.47
Non-human primates	various	various
Chimpanzee	420	0.97
Gorilla	465	1.05
Orangutan	370	1.09
Mandrill	87	1.02

Example question: Construct a 95% CI for the predicted glia-neuron ratio for humans and interpret whether it is unusually high compared to other primates.

Summary Table: Types of Confidence Intervals

Interval Type	Purpose	Formula
CI for Mean Response	Estimate average value at given x	$\hat{y} \pm t^* \cdot SE_{mean}$
CI for Individual Response	Estimate value for individual at given x	$\hat{y} \pm t^* \cdot SE_{indiv}$

Key Takeaways

• Confidence intervals quantify uncertainty in parameter estimates and predictions.

• Model assumptions must be checked using residual plots and transformations.

• ANOVA is used to compare group means and requires checking assumptions.

• Model selection should balance statistical validity, fit, simplicity, and biological relevance.

Additional info: These notes expand on the lecture slides by providing definitions, formulas, and context for statistical inference in regression and ANOVA, suitable for college-level statistics students.