Linear Regression Using the Least Squares Method: Study Notes

Linear Regression Using the Least Squares Method

Introduction to Least Squares Regression

Linear regression is a fundamental statistical technique used to model the relationship between two quantitative variables. The least squares method is employed to find the best-fitting line through a set of data points by minimizing the sum of the squared residuals.

• Linear Regression: A method to model the relationship between two variables with an equation of a line: $y = mx + b$.

• Residual: The difference between an observed value and the value predicted by the regression line. $\text{Residual} = y_{\text{observed}} - y_{\text{predicted}}$

• Least Squares Regression: Minimizes the sum of squared residuals to determine the best-fitting line.

Example: The data below shows ice cream sales (y) in dollars from a local ice cream stand and the daily high temperature (x) in °F for 1 week. Find and plot the least squares regression line for the data. Once you plot the line, use the data to interpret the fit and answer questions about predictions.

High Temp (°F)	Sales ($)
74	2842
82	4192
86	4752
90	5243
94	5843
98	6397

Additional info: The regression line can be found using statistical software or a calculator. The equation will have the form $\hat{y} = a + bx$, where $a$ is the intercept and $b$ is the slope.

How to Find the Least Squares Regression Line on a TI-84 Calculator

Follow these steps to compute the regression line using a TI-84 calculator:

1. Enter data into STAT → EDIT

2. Press STAT → CALC

3. Select LinReg(ax+b)

4. Enter L1, L2 (or your data lists)

5. Press ENTER to view the regression equation

Interpreting Regression Equations and Scatterplots

Identifying the Regression Line Equation

Given a scatterplot and a regression line, you can match the equation to the graph by examining the slope and intercept.

• Slope (b): Indicates the rate of change of y with respect to x.

• Intercept (a): The predicted value of y when x = 0.

Example: A scatterplot shows a set of data and its least-squares regression line. Based on the graph, select the equation that best fits the regression line from a list of options.

Predicting Values with Regression Lines

Using the Regression Line for Prediction

The regression line can be used to predict the value of the dependent variable (y) for a given value of the independent variable (x), provided the correlation is strong and the value of x is within the range of the data.

• If correlation is strong and x-value is within the range of data, use the regression equation to predict y.

• If correlation is weak or x-value is outside the range of data, use the regression equation with caution (may not be reliable).

Example: Using the ice cream sales and temperature data, predict sales for a high temperature of 77°F. Use the regression equation to calculate the predicted sales value.

Practice: Applying Regression Analysis

Regression Equation for Sales Data

A regional sales manager records data on the number of clients a salesperson contacts in a week (x) and the total sales generated that week (y). The data from 10 salespeople is shown below. Find the equation of the regression line and use it to predict sales if the salesperson contacts (a) 6 clients, (b) 43 clients.

Clients (x)	Sales ($)
2	750
10	1800
18	2700
26	3500
34	3900
38	4000
40	5000
43	5100

Additional info: The regression equation can be calculated using the least squares method. For prediction, substitute the desired x-value into the regression equation.

Summary Table: Key Concepts in Least Squares Regression

Concept	Definition	Application
Linear Regression	Models relationship between two variables with a straight line	Predicting y from x
Least Squares Method	Minimizes sum of squared residuals	Finding best-fit line
Residual	Difference between observed and predicted y	Assessing fit quality
Correlation	Strength and direction of linear relationship	Determining reliability of predictions