Section 9.2 – Linear Regression: Regression Lines, Residuals, and Prediction

Linear Regression

Introduction to Linear Regression

Linear regression is a statistical method used to model the relationship between two quantitative variables by fitting a straight line (the regression line) to the observed data. The goal is to minimize the differences between the observed values and the values predicted by the line.

• Regression line (line of best fit): A straight line that best describes how the response variable y changes as the explanatory variable x changes.

• Equation of the regression line: $\hat{y} = mx + b$

• ŷ (y-hat): Predicted value of y for a given x-value.

• m: Slope of the line (rate of change of y with respect to x).

• b: y-intercept (value of y when x = 0).

Additional info: The regression line is determined using all data points, not just two as in algebra. It is used for prediction only when there is a significant correlation between x and y.

Residuals

Residuals measure the vertical distance between the observed y-values and the predicted y-values on the regression line. They indicate how well the regression line fits the data.

• Residual (d): $d = \text{(observed y-value)} - \text{(predicted y-value)}$

• A small residual means the prediction is accurate; a large residual means the prediction is less accurate.

Additional info: Residuals are used to assess the goodness of fit of the regression model. The best-fitting line minimizes the sum of the squared residuals.

Interpreting the Regression Line

The regression line is meaningful only when the correlation between x and y is strong or significant. The sign of the slope (m) indicates the direction of the relationship:

• Positive slope: Indicates a positive linear correlation (as x increases, y increases).

• Negative slope: Indicates a negative linear correlation (as x increases, y decreases).

Predictions using the regression line are only meaningful for x-values within the range of the original data (interpolation). Predictions outside this range (extrapolation) are generally unreliable.

Formulas for the Regression Line

To find the regression line, follow these steps:

1. Find the slope (m):

$m = \frac{ [ n \ast \sum xy ] - [ \sum x \ast \sum y ] }{ [ n \ast \sum x^2 ] - (\sum x)^2 }$

2. Find the y-intercept (b):

$b = \left( \frac{ \sum y }{ n } \right) - \left( m \ast \frac{ \sum x }{ n } \right)$

3. Write the regression equation: $\hat{y} = mx + b$

When substituting values for m and b, round to the thousandths place.

Using the Regression Equation for Prediction

To predict the value of y for a given x-value, substitute x into the regression equation. Ensure that the x-value is within the range of the original data for the prediction to be meaningful.

• If x is outside the range, state: "It is not meaningful to predict the value of y for x = (value) because x = (value) is outside the range of the original data."

Worked Example: Regression Line Calculation

Given data: Heights (x) and number of stories (y) for nine buildings in Atlanta.

• $\sum x = 6158$

• $\sum y = 405$

• $\sum xy = 286,054$

• $\sum x^2 = 4,350,854$

• $n = 9$

Find the slope (m):

$m = \frac{ [9 \ast 286,054] - [6158 \ast 405] }{ [9 \ast 4,350,854] - (6158)^2 } = \frac{2,574,486 - 2,493,990}{39,157,686 - 37,920,964} = \frac{80,496}{1,236,722} \approx 0.065088$

Find the y-intercept (b):

$b = \left( \frac{405}{9} \right) - \left( 0.065088 \ast \frac{6158}{9} \right) \approx 0.465$

Regression equation: $\hat{y} = 0.065x + 0.465$

Prediction Example

• For x = 800 feet: $\hat{y} = 0.065 \ast 800 + 0.465 = 52.465$ (meaningful, since 800 is within the data range)

• For x = 400 feet: Not meaningful, since 400 is outside the data range (486 to 869).

Correlation Coefficient (from Section 9.1)

The sample correlation coefficient r measures the strength and direction of the linear relationship between x and y.

$r = \frac{ [ n \ast \sum xy ] - [ \sum x \ast \sum y ] }{ \sqrt{ [ n \ast \sum x^2 ] - (\sum x)^2 } \ast \sqrt{ [ n \ast \sum y^2 ] - (\sum y)^2 } }$

Interpretation: r close to 1 or -1 indicates a strong linear relationship; r close to 0 indicates a weak relationship.

Summary Table: Steps for Finding the Regression Line

Step	Description	Formula
1	Find the slope (m)	$m = \frac{ [ n \ast \sum xy ] - [ \sum x \ast \sum y ] }{ [ n \ast \sum x^2 ] - (\sum x)^2 }$
2	Find the y-intercept (b)	$b = \left( \frac{ \sum y }{ n } \right) - \left( m \ast \frac{ \sum x }{ n } \right)$
3	Write the regression equation	$\hat{y} = mx + b$
4	Predict y for a given x	Substitute x into $\hat{y} = mx + b$