Describing the Relation Between Two Variables: Scatter Diagrams, Correlation, and Regression

Describing the Relation Between Two Variables

Scatter Diagrams and Types of Association

Scatter diagrams are essential tools in statistics for visualizing the relationship between two quantitative variables measured on the same individuals. The predictor (independent) variable is plotted on the horizontal axis, while the response (dependent) variable is plotted on the vertical axis. Each point represents an individual observation.

• Positively Associated Variables: When increases in the predictor variable are associated with increases in the response variable.

• Negatively Associated Variables: When increases in the predictor variable are associated with decreases in the response variable.

• No Association: When there is no discernible pattern between the variables.

Interpretation: Scatter diagrams help identify whether the relationship is linear, nonlinear, or if no relationship exists. Linear relationships can be positive or negative, while nonlinear relationships may show curves or clusters.

Correlation Coefficient

The correlation coefficient quantifies the strength and direction of a linear relationship between two quantitative variables. The population correlation coefficient is denoted by $\rho$ (rho), and the sample correlation coefficient by $r$.

• Formula for the Sample Correlation Coefficient:

$r = \frac{\sum (x_i - \overline{x})(y_i - \overline{y})}{(n-1)s_x s_y}$ Alternatively, $r = \frac{n\sum x_i y_i - \sum x_i \sum y_i}{\sqrt{[n\sum x_i^2 - (\sum x_i)^2][n\sum y_i^2 - (\sum y_i)^2]}}$

• Properties of r:

  • -1 ≤ r ≤ 1

  • r = +1: Perfect positive linear relation

  • r = -1: Perfect negative linear relation

  • r close to 0: No linear relation

  • r is unitless

Example: For the productivity-experience data, r = 0.96 indicates a strong positive linear relationship.

Application: Software such as Excel can be used to compute the correlation coefficient efficiently.

Least-Squares Regression

Regression analysis estimates the relationship between a response variable and a predictor variable by fitting a line that best represents the data. The least-squares regression line minimizes the sum of squared vertical distances (errors) between observed and predicted values.

• Population Model: $y_i = \beta_0 + \beta_1 x_i + \varepsilon_i$

• Sample Model: $y_i = b_0 + b_1 x_i + u_i$

• Least-Squares Regression Equation: $\hat{y} = b_0 + b_1 x$

• Formulas for Coefficients:

  • $b_1 = \frac{n\sum x_i y_i - \sum x_i \sum y_i}{n\sum x_i^2 - (\sum x_i)^2}$

  • $b_0 = \overline{y} - b_1 \overline{x}$

  • Alternatively, $b_1 = r \frac{s_y}{s_x}$

Interpretation: The slope $b_1$ represents the change in the response variable for each unit increase in the predictor variable. The intercept $b_0$ is the predicted value of y when x = 0 (if meaningful within the data range).

Prediction and Scope of the Model

The regression equation can be used to predict the response variable for given values of the predictor variable, but only within the range of observed data. Predictions outside this range (extrapolation) are unreliable.

• Prediction Equation: $\hat{y} = b_0 + b_1 x$

• Residual (Error): $u_i = y_i - \hat{y}_i$

Example: For a worker with 7 years of experience, predicted productivity is $\hat{y} = 8.5 + 4.2 \times 7$.

Measuring the Fit: Coefficient of Determination (R2)

The coefficient of determination, $R^2$, measures the proportion of total variation in the response variable explained by the regression line.

• $0 \leq R^2 \leq 1$

• $R^2 = r^2$ for simple linear regression

• Interpretation: An $R^2$ of 0.92 means 92% of the variation in productivity is explained by experience.

Deviations:

• Total deviation: $y - \overline{y}$

• Explained deviation: $\hat{y} - \overline{y}$

• Unexplained deviation: $y - \hat{y}$

Standard Error of the Estimate

The standard error of the estimate, $s_e$, measures the typical distance that the observed values fall from the regression line.

• $s_e = \sqrt{\frac{\sum (y_i - \hat{y}_i)^2}{n-2}}$

• Smaller $s_e$ indicates a better fit; $s_e = 0$ means all points lie exactly on the regression line.

Hypothesis Testing for the Slope Coefficient

Hypothesis testing determines whether there is a statistically significant linear relationship between the predictor and response variables.

• Null Hypothesis: $H_0: \beta_1 = 0$ (no linear relation)

• Alternative Hypothesis: $H_1: \beta_1 \neq 0$ (two-tailed), $H_1: \beta_1 < 0$ (left-tailed), $H_1: \beta_1 > 0$ (right-tailed)

• Test Statistic: $t = \frac{b_1 - \beta_1}{s_{b_1}}$, where $s_{b_1} = \frac{s_e}{\sqrt{\sum (x_i - \overline{x})^2}}$

• Degrees of freedom: $n - 2$

Decision Rule: Reject $H_0$ if the calculated t-value falls in the rejection region determined by the significance level $\alpha$.

Example: For the productivity-experience data, $t = 5.93$ exceeds the critical value $t_{0.025,3} = 3.182$, so we reject $H_0$ and conclude a significant positive relationship exists.

Using Excel for Correlation and Regression Analysis

Excel provides tools for calculating correlation coefficients and fitting regression models.

• Correlation: Use the Data Analysis Toolpak, select 'Correlation', input the data range, and specify output options.

• Regression: Use the Data Analysis Toolpak, select 'Regression', input the Y and X ranges, and specify output options.

Summary Table: Key Formulas and Concepts

Concept	Formula/Definition
Correlation Coefficient (r)	$r = \frac{\sum (x_i - \overline{x})(y_i - \overline{y})}{(n-1)s_x s_y}$
Regression Line	$\hat{y} = b_0 + b_1 x$
Slope (b1)	$b_1 = r \frac{s_y}{s_x}$
Intercept (b0)	$b_0 = \overline{y} - b_1 \overline{x}$
Coefficient of Determination (R2)	$R^2 = r^2$
Standard Error of Estimate (se)	$s_e = \sqrt{\frac{\sum (y_i - \hat{y}_i)^2}{n-2}}$
Test Statistic for Slope	$t = \frac{b_1 - \beta_1}{s_{b_1}}$