Regression Analysis: Interpreting Slope, Intercept, Model Fit, and Residuals

Regression Analysis

Interpreting the Slope and Intercept

In simple linear regression, the slope and intercept of the regression line have important interpretations. The slope indicates the expected change in the response variable for a one-unit increase in the predictor variable, while the intercept represents the expected value of the response when the predictor is zero (if meaningful in context).

• Slope (b1): Change in the predicted response per unit increase in the predictor.

• Intercept (b0): Predicted response when the predictor is zero.

Example: If the regression equation for drug dose and concentration is y = 2.1x + 0.5, then for each additional mg of drug, the mean concentration increases by 2.1 units.

Inference for the Regression Slope

Statistical inference allows us to test whether the slope is significantly different from zero and to estimate confidence intervals for the slope.

• Hypothesis Test: Test if the slope is zero (no linear association) or not.

• Test Statistic: $t = \frac{b_1}{SE_{b_1}}$ where $SE_{b_1}$ is the standard error of the slope.

• Confidence Interval for Slope: $b_1 \pm t^* SE_{b_1}$ where $t^*$ is the critical value from the t-distribution.

Example: In a study of enzyme concentration and reaction rate, a 95% confidence interval for the slope might be (0.33, 0.53), indicating a significant positive association.

Assessing the Fit

Assessing model fit involves measuring how well the regression line explains the variability in the response variable. Two key statistics are the sum of squares due to regression (SSR) and the sum of squared errors (SSE):

• $SSR = \sum (\hat{y}_i - \bar{y})^2$

• $SSE = \sum (y_i - \hat{y}_i)^2$

The coefficient of determination ($R^2$) measures the proportion of variation in the response explained by the model:

• $R^2 = \frac{SSR}{SST} = 1 - \frac{SSE}{SST}$

Root Mean Square Error (RMSE): The square root of the average squared residuals, representing the typical size of prediction errors.

Residual Analysis

Residual analysis checks the assumptions of linear regression by examining the differences between observed and predicted values.

• Linearity: Relationship between x and y should be linear.

• Independence of errors: Residuals should be independent.

• Normality of errors: Residuals should be approximately normally distributed.

• Equal variance (Homoscedasticity): Residuals should have constant variance across all levels of the predictor.

Residual plots are used to visually check these assumptions. Patterns or non-constant spread in residual plots may indicate violations.

Prediction Intervals and Confidence Intervals

Regression analysis provides both confidence intervals for the mean response and prediction intervals for individual responses at a given value of the predictor.

• Residual Standard Deviation (s): $s = \sqrt{\frac{\sum (y_i - \hat{y}_i)^2}{n-2}}$

• Confidence Interval for Mean Response: $\hat{y}_h \pm t^* s \sqrt{\frac{1}{n} + \frac{(x_h - \bar{x})^2}{S_{xx}}}$

• Prediction Interval for Individual Response: $\hat{y}_h \pm t^* s \sqrt{1 + \frac{1}{n} + \frac{(x_h - \bar{x})^2}{S_{xx}}}$

Prediction intervals are wider than confidence intervals because they account for both the uncertainty in estimating the mean and the variability of individual outcomes.

Summary Table: Key Regression Concepts

Concept/Statistic	Interpretation
Sample Slope (b1)	Change in the predicted response per unit increase in the predictor
Sample Intercept (b0)	Predicted response when the predictor is zero
t test for Slope	Test if slope is significantly different from zero
Confidence Interval for Slope	Range of plausible values for the slope
Coefficient of Determination (R2)	Fraction of variation in response explained by the model
Root Mean Square Error (RMSE)	Typical size of prediction errors in the response variable's units

Additional info:

• Examples and applications are provided for interpreting regression coefficients in various scientific contexts (e.g., drug dose, blood pressure, enzyme kinetics).

• Instructions for using JMP software to perform regression inference and residual analysis are included.

• Common pitfalls, such as extrapolation beyond the data range, are highlighted.