Exploring Relationships Between Variables: Scatterplots, Correlation, and Linear Regression

Exploring Relationships Between Variables

Scatterplots and the Correlation Coefficient

Scatterplots are a fundamental graphical tool in statistics for visualizing the relationship between two quantitative variables. They help identify patterns, trends, and possible associations between variables.

• Definition: A scatterplot is a graph in which each point represents a pair of values for two variables, typically labeled X (predictor) and Y (response).

• Example: The relationship between monthly advertising expenditures (in thousands of dollars) and total sales (in thousands of dollars) for a brewery is shown below.

Month	Advertising	Sales
1	2.2	27.5
2	1.6	26.9
3	2.9	29.8
4	3.2	29.2
5	3.5	30.3
6	3.4	30.0
7	2.3	29.1
8	2.5	28.3
9	1.0	26.5
10	2.5	28.9
11	3.4	30.4
12	0.9	26.8
13	0.9	26.6
14	1.4	26.9
15	2.6	28.2

• Interpretation: The scatterplot of these data shows that as advertising expenditures increase, total sales tend to increase, indicating a positive linear relationship.

How to Interpret a Scatterplot

Interpretation of a scatterplot focuses on the direction and strength of the relationship between variables.

• Positive Linear Relationship: As X increases, Y tends to increase.

• Negative Linear Relationship: As X increases, Y tends to decrease.

• No Linear Relationship: As X increases, Y tends to neither increase nor decrease.

• Note: Always interpret scatterplots in terms of the actual variable names, not just X and Y.

• Example: A scatterplot of EPA gas mileage ratings (MPG) versus horsepower for car models shows a negative linear relationship: as horsepower increases, MPG tends to decrease.

The Correlation Coefficient (r)

The correlation coefficient quantifies the strength and direction of a linear relationship between two quantitative variables.

• Definition: The correlation coefficient, denoted as $r$, measures the degree to which two variables are linearly related.

• Properties:

  • The value of $r$ is always between -1 and 1.

  • The sign of $r$ matches the trend of the scatterplot: positive for upward, negative for downward, zero for no trend.

  • If all points fall perfectly on a straight line with an upward trend, $r = 1$.

  • If all points fall perfectly on a straight line with a downward trend, $r = -1$.

  • If the relationship is very weak or non-linear, $r$ is near 0.

• Visual Examples:

  • $r = +1$: Perfect positive linear relationship

  • $r = -1$: Perfect negative linear relationship

  • $r$ near 0: Very weak or no linear relationship

  • $r$ near +1: Strong positive linear relationship

  • $r$ near -1: Strong negative linear relationship

Testing for a Correlation Between Two Quantitative Variables

Statistical hypothesis testing can determine whether a significant linear relationship exists between two variables.

• Null Hypothesis ($H_0$): X and Y are not correlated.

• Alternative Hypothesis ($H_a$): X and Y are correlated.

• Test Statistic: $r$

• Decision Rule: Accept $H_a$ if the p-value $< \alpha$ (commonly $\alpha = 0.05$).

• Example: For advertising expenditure and sales, $r = 0.9475$, p-value $< 0.0001$. Since p-value $< 0.05$, we reject $H_0$ and conclude a positive linear relationship exists.

Fitting a Straight Line to Sampled Data: The Least Squares Prediction Equation

General Form of the Least Squares Line

The least squares regression line is the best-fitting straight line through a set of data points, minimizing the sum of squared vertical distances from the points to the line.

• Equation: $Y = a + bX$

• Where:

  • $a$ = Y-intercept (value of Y when X = 0)

  • $b$ = slope (change in Y for a one-unit increase in X)

• Example: For the brewery data, the prediction equation is:

  • $\text{Sales} = 25.051529 + 1.446853 \times \text{Advertising}$

• Interpretation: The slope ($b$) indicates the expected increase in sales for each additional thousand dollars spent on advertising. The intercept ($a$) estimates sales when advertising is zero.

• Prediction: If the brewery spends $2,000 on advertising:

  • $\text{Sales} = 25.051529 + 1.446853 \times 2 = 27.945236$

Describing the Strength of the Linear Relationship: Coefficient of Determination ($r^2$)

The coefficient of determination ($r^2$) quantifies the proportion of variance in the response variable explained by the predictor variable.

• Definition: $r^2$ is the fraction of the variability in Y explained by a linear relationship with X.

• Properties:

  • $r^2$ is always between 0 and 1.

  • $1 - r^2$ is the fraction of variability in Y not explained by X (unexplained or due to error).

• Example: If $r^2 = 0.8977$, then 89.77% of the variation in monthly sales is explained by advertising expenditures.

Prediction and Confidence Intervals

Predicted values from the regression line can be interpreted in two ways:

• Estimated Mean Sales: The average sales for all months with a given advertising expenditure.

• Predicted Sales for a Single Month: The expected sales for a specific month with a given advertising expenditure.

• Confidence Interval: Used to estimate the mean value of Y for a particular value of X.

• Prediction Interval: Used to predict a single value of Y for a particular value of X.

Additional info: Confidence and prediction intervals are important for quantifying uncertainty in regression predictions.