Scatter Diagrams and Correlation: Study Notes for Statistics Students

Scatter Diagrams and Correlation

Introduction

This section explores the graphical and numerical methods for describing the relationship between two quantitative variables. The focus is on scatter diagrams, the linear correlation coefficient, and the distinction between correlation and causation.

Response and Explanatory Variables

Definitions

• Response variable: The variable you want to predict. Also called the dependent variable.

• Explanatory (or predictor) variable: The variable used to predict the response variable. Also called the independent variable.

In a statistical relationship, the response variable's value is explained by the value of the explanatory variable.

Scatter Diagrams

Definition and Construction

• Scatter diagram: A graph showing the relation between two quantitative variables measured on the same individual.

• Each data point represents a pair of observations: $(x_1, y_1), \ldots, (x_n, y_n)$.

• Procedure:

  1. Plot each data point on the diagram.

  2. Response variable (y): vertical axis.

  3. Explanatory variable (x): horizontal axis.

Example

A golf pro investigates the relationship between club-head speed (mph) and driving distance (yards):

Club-head Speed (mph)	Distance (yards)
100	257
102	264
103	274
101	266
105	277
106	263
99	258
103	275

The scatter diagram shows that as club-head speed increases, the distance the ball travels also increases.

Types of Relationships in Scatter Diagrams

• Linear relation: Points form a pattern that can be approximated by a straight line.

• Nonlinear relation: Points form a pattern that is not linear (e.g., curved).

• No relation: Points are scattered randomly, with no apparent pattern.

Positive vs Negative Association

• Positive association: As one variable increases, the other also increases.

• Negative association: As one variable increases, the other decreases.

Linear Functions

Definition

• A linear function is given by the formula $y = f(x) = ax + b$.

• The graph of a linear function is a straight line.

• a: slope of the line; b: y-intercept.

Linear Correlation Coefficient

Definition and Purpose

• The linear correlation coefficient (Pearson correlation coefficient) measures the strength and direction of the linear relationship between two quantitative variables.

• Population correlation coefficient: $\rho$ (rho).

• Sample correlation coefficient: $r$.

Formulas

• Sample covariance between $x$ and $y$: $s_{xy} = \text{Cov}(x, y) = \frac{1}{n-1} \sum (x_i - \overline{x})(y_i - \overline{y})$

• Sample linear correlation coefficient: $r = \frac{s_{xy}}{s_x s_y}$ where $s_x$ and $s_y$ are the sample standard deviations of $x$ and $y$.

• Expanded formula: $r = \frac{\frac{1}{n-1} \sum (x_i - \overline{x})(y_i - \overline{y})}{\sqrt{\frac{1}{n-1} \sum (x_i - \overline{x})^2} \sqrt{\frac{1}{n-1} \sum (y_i - \overline{y})^2}}$

Properties of the Linear Correlation Coefficient

• $-1 \leq r \leq 1$

• If $r = +1$, perfect positive linear relation.

• If $r = -1$, perfect negative linear relation.

• If $r$ is close to 0, little or no evidence of a linear relation (but possibly a nonlinear relation).

• Not resistant: Outliers can greatly affect $r$.

• Unitless: The units of $x$ and $y$ do not affect $r$.

Interpreting Scatter Plots and Correlation Coefficient

• Perfect positive: $r = +1$

• Strong positive: $r$ close to +1

• Moderate positive: $r$ moderately positive

• Perfect negative: $r = -1$

• Strong negative: $r$ close to -1

• Moderate negative: $r$ moderately negative

• No linear relation: $r \approx 0$

Testing for a Linear Relation

Steps

1. Determine the absolute value of the correlation coefficient.

2. Find the critical value for the given sample size $n$ (see Table II below).

3. If $|r|$ is greater than the critical value, a linear relation exists; otherwise, no linear relation exists.

Table: Critical Values for Correlation Coefficient

n	Critical Value
3	0.997
4	0.990
5	0.878
6	0.811
7	0.754
8	0.707
9	0.666
10	0.632
11	0.602
12	0.576
13	0.553
14	0.532

Worked Example: Correlation between BMI and Blood Pressure

Data Table

x = Body Mass Index	y = Systolic Blood Pressure
18.4	120
20.1	110
22.4	120
25.9	135
26.5	140
28.9	115
30.1	150
32.9	165
33.0	160
34.7	180

Calculation Steps

• Compute variance of $x$: $s_x^2 = \frac{\sum (x_i - \overline{x})^2}{n-1} = 31.852$

• Compute variance of $y$: $s_y^2 = \frac{\sum (y_i - \overline{y})^2}{n-1} = 563.611$

• Compute covariance: $s_{xy} = \frac{\sum (x_i - \overline{x})(y_i - \overline{y})}{n-1} = 115.22$

• Compute $r$: $r = \frac{115.22}{\sqrt{31.852 \times 563.611}} = 0.859$

• Compare to critical value for $n = 10$: $0.632$

• Since $0.859 > 0.632$, there is a positive linear relation between BMI and SBP.

Conclusion: BMI and SBP are positively correlated in this sample.

Outliers and Correlation

Effect of Outliers

• Outliers can distort the value of the correlation coefficient, making it appear weaker or stronger than it truly is.

• Always inspect scatter plots for outliers before interpreting $r$.

Correlation vs Causation

Key Distinctions

• Correlation measures association, not causation.

• High correlation may result from both variables increasing over time (e.g., time series data).

• Causality can only be claimed when data are collected through a designed experiment, not from observational data.

Example

A correlation of 0.940 between the percentage of females with bachelor's degrees and the percentage of births to unmarried mothers does not imply that one causes the other; both may be increasing over time independently.

Practice and Application

Exercises

• Given a scatterplot, estimate whether the correlation is positive, negative, or near zero.

• Calculate $r$ for a given data set.

• Consider how changes in units or outliers affect the correlation coefficient.

Summary Table: Types of Relationships in Scatter Diagrams

Type	Description	Correlation Coefficient
Perfect positive linear	All points lie on a straight line with positive slope	$r = +1$
Strong positive linear	Points closely follow a line with positive slope	$r$ close to +1
Moderate positive linear	Points loosely follow a line with positive slope	$r$ moderately positive
Perfect negative linear	All points lie on a straight line with negative slope	$r = -1$
Strong negative linear	Points closely follow a line with negative slope	$r$ close to -1
Moderate negative linear	Points loosely follow a line with negative slope	$r$ moderately negative
No linear relation	Points scattered randomly	$r \approx 0$
Nonlinear relation	Points follow a curved pattern	$r \approx 0$

Additional info: These notes expand on the original lecture slides by providing full definitions, formulas, and context for the interpretation and calculation of the linear correlation coefficient, as well as the distinction between correlation and causation. All tables have been recreated in HTML for clarity and completeness.