Linear Regression with Two Quantitative Variables

Introduction to Linear Regression

When analyzing the relationship between two quantitative variables, a model is an equation that best summarizes how the variables are related. Linear regression is a statistical method used to model and analyze the relationship between a dependent variable and an independent variable by fitting a linear equation to observed data.

• Model: An equation that describes the relationship between variables, used for prediction.

• Example: Forecasting the weather or predicting manufacturing costs.

• Linear models are used when the rate of change between variables is constant.

Rate of Change and When to Use Linear Models

The rate of change (ROC) refers to how much the dependent variable (y) changes for each one-unit increase in the independent variable (x). If this rate is constant, a linear model is appropriate.

• Constant Rate of Change: The y-values change by the same amount for each unit increase in x.

• Non-constant Rate of Change: The change in y varies as x increases (e.g., economies of scale in manufacturing).

• Use a linear model only when the rate of change is constant.

Regression Line and the Least Squares Method

A regression line summarizes the linear relationship between two variables in a scatterplot. The least-squares regression (LSR) line is the line that minimizes the sum of the squares of the residuals (differences between observed and predicted values).

• Regression Line: A straight line that best fits the data points in a scatterplot.

• Least Squares Regression Line: The line that minimizes the sum of squared residuals.

• Technology (e.g., StatKey) is often used to compute the slope and intercept of the LSR line.

Equation of a Regression Line

All nonvertical lines can be written in the slope-intercept form:

• General form:

• : The y-intercept (the value of y when x = 0).

• : The slope (the constant rate of change of y with respect to x).

• In some contexts, is used, but in statistics, is standard.

Interpreting the Regression Line

The regression line is used to predict the value of the response variable (y) from a given value of the explanatory variable (x).

• Observed Value (y): The actual value measured in the data.

• Predicted Value (): The value predicted by the regression line for a given x.

• Residual: The difference between the observed and predicted values:

Example: Cricket Chirps and Temperature

• Regression equation:

• If crickets chirp 150 times per minute: °F

• If the observed temperature is 72 °F, the residual is °F

• Interpretation: The observed temperature is 0.239 °F less than the predicted value.

Interpreting the Slope and Intercept

• Y-intercept (): The predicted value of y when x = 0. Its interpretation depends on whether x = 0 is meaningful in context.

• Example: In the cricket chirps example, the y-intercept predicts the temperature when crickets are not chirping. However, if x = 0 is not possible or not observed in the data, the y-intercept may be meaningless or unreliable.

• Slope (): The amount by which the predicted y changes for each one-unit increase in x.

• Example: In a car mileage dataset, a slope of 1.687 means that for each 1 mpg increase in city fuel efficiency, highway fuel efficiency is predicted to increase by 1.687 mpg.

Extrapolation

Extrapolation refers to making predictions for x-values that are far outside the range of the data used to fit the regression line. Such predictions are often unreliable.

• Predictions are most reliable within the range of observed x-values.

• Extrapolating far beyond the data can lead to misleading or meaningless results.

Regression Cautions

• Avoid Extrapolation: Do not use the regression line to predict y for x-values far outside the observed range.

• Check Linearity: Always plot the data to ensure a linear trend before fitting a regression line.

• Outliers and Influential Points: Outliers, especially those with extreme x-values, can have a strong influence on the regression line. A point is influential if removing it causes large changes in the regression equation.

• Compare regression lines with and without suspected outliers to assess their influence.

Summary Table: Key Terms in Linear Regression

Additional info: In practice, always use scatterplots to visually assess linearity and check for outliers before interpreting regression results. The least-squares regression line is foundational for more advanced statistical modeling, including multiple regression and inference about regression parameters.