Chapter 12: Simple Linear Regression – Study Notes

Simple Linear Regression

Introduction to Regression Analysis

Regression analysis is a fundamental statistical technique used in business to model and analyze the relationship between variables. In simple linear regression, the goal is to predict the value of a dependent variable (Y) based on the value of a single independent variable (X).

• Regression analysis helps to:

  • Predict the value of a dependent variable from one or more independent variables.

  • Explain how changes in an independent variable affect the dependent variable.

• Dependent variable (Y): The variable to be predicted or explained.

• Independent variable (X): The variable used to predict or explain the dependent variable.

Preliminary Analysis – Scatter Plots

A scatter plot is a graphical tool used to visualize the relationship between two quantitative variables, X and Y. It is often the first step in regression analysis.

• Helps identify the type of relationship (linear, curvilinear, or no relationship).

• Suggests whether regression analysis is appropriate.

Types of Relationships

• Linear Relationship: Data points form a straight line pattern.

• Curvilinear Relationship: Data points form a curved pattern.

• No Relationship: Data points are scattered with no discernible pattern.

Simple Linear Regression Model

Simple linear regression models the relationship between Y and X using a linear equation:

• Only one independent variable (X).

• Assumes changes in Y are linearly related to changes in X.

Population regression equation:

• $Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i$

• Where $\beta_0$ is the intercept, $\beta_1$ is the slope, and $\varepsilon_i$ is the random error term.

Estimated regression equation (prediction line):

• $\hat{Y}_i = b_0 + b_1 X_i$

• $b_0$ and $b_1$ are sample estimates of $\beta_0$ and $\beta_1$.

The Least Squares Method

The least squares method determines the best-fitting regression line by minimizing the sum of squared differences between observed and predicted values.

• Objective: $\min \sum (Y_i - \hat{Y}_i)^2 = \min \sum (Y_i - (b_0 + b_1 X_i))^2$

• Regression coefficients $b_0$ and $b_1$ are calculated to achieve this minimum.

Interpretation of the Slope and Intercept

• Intercept ($b_0$): Estimated mean value of Y when X = 0.

• Slope ($b_1$): Estimated change in mean value of Y for a one-unit increase in X.

Worked Example: Home Prices and Square Footage

A real estate agent examines the relationship between house price (Y, in $1,000s) and house size (X, in square feet) using a sample of 10 houses.

House Price in $1000s (Y)	Square Feet (X)
245	1400
312	1600
279	1700
308	1875
199	1100
219	1550
405	2350
324	2450
319	1425
255	1700

Scatter plot and regression analysis are performed to estimate the relationship.

Regression Output and Interpretation

• Regression equation (Excel output): house price = 98.24833 + 0.10977 × (square feet)

• Interpretation of $b_0$: The estimated mean price when square feet = 0 (not meaningful in this context).

• Interpretation of $b_1$: For each additional square foot, the mean house price increases by $109.77.

Making Predictions

• To predict the price of a house with 2,000 square feet:

$\text{house price} = 98.25 + 0.1098 \times 2000 = 317.85$ ($1,000s) = $317,850

• Only predict within the range of observed X values (do not extrapolate).

Computing $b_0$ and $b_1$

• For small datasets, $b_0$ and $b_1$ can be calculated by hand:

$b_1 = \frac{SSXY}{SSX}$

$b_0 = \overline{Y} - b_1 \overline{X}$

Where:

• $SSXY = \sum (X_i - \overline{X})(Y_i - \overline{Y}) = \sum X_i Y_i - \frac{\sum X_i \sum Y_i}{n}$

• $SSX = \sum (X_i - \overline{X})^2 = \sum X_i^2 - \frac{(\sum X_i)^2}{n}$

• $\overline{Y} = \frac{\sum Y_i}{n}$, $\overline{X} = \frac{\sum X_i}{n}$

Measures of Variation

Total variation in Y is partitioned into explained and unexplained components:

$SST = SSR + SSE$

• SST (Total Sum of Squares): $\sum (Y_i - \overline{Y})^2$

• SSR (Regression Sum of Squares): $\sum (\hat{Y}_i - \overline{Y})^2$

• SSE (Error Sum of Squares): $\sum (Y_i - \hat{Y}_i)^2$

Coefficient of Determination ($r^2$)

The coefficient of determination measures the proportion of total variation in Y explained by X.

$r^2 = \frac{SSR}{SST}$

• Ranges from 0 to 1.

• $r^2 = 1$ indicates a perfect linear relationship.

• $0 < r^2 < 1$ indicates a weaker relationship.

• $r^2 = 0$ indicates no linear relationship.

Summary Table: Key Regression Quantities

Symbol	Name	Formula	Interpretation
$b_0$	Intercept	$\overline{Y} - b_1 \overline{X}$	Estimated mean Y when X = 0
$b_1$	Slope	$\frac{SSXY}{SSX}$	Change in mean Y per unit X
$SST$	Total Sum of Squares	$\sum (Y_i - \overline{Y})^2$	Total variation in Y
$SSR$	Regression Sum of Squares	$\sum (\hat{Y}_i - \overline{Y})^2$	Variation explained by X
$SSE$	Error Sum of Squares	$\sum (Y_i - \hat{Y}_i)^2$	Unexplained variation
$r^2$	Coefficient of Determination	$\frac{SSR}{SST}$	Proportion of explained variation

Additional info:

• These notes are based on textbook slides and cover the essential concepts, formulas, and interpretation for simple linear regression in business statistics.

• Further topics such as hypothesis testing for regression coefficients, confidence intervals, and residual analysis are typically included in a full chapter but are not shown in these slides.