Exploring Two-Variable Data: Categorical and Quantitative Relationships

Exploring Two-Variable Data

Comparing Two Categorical Variables

When analyzing two categorical variables, statisticians use tables and graphs to summarize relationships and calculate relevant statistics. Understanding independence and relative risk is crucial for interpreting these associations.

• Two-Way Tables: Organize data for two categorical variables, showing frequencies for each combination of categories.

• Marginal Relative Frequencies: Calculated by dividing row or column totals by the overall total. These represent the proportion of observations in each category.

• Conditional Relative Frequencies: Calculated by dividing cell frequencies by the total for a specific row or column. These show the proportion within a subgroup.

• Segmented Bar Graphs: Used to visually compare two categorical variables. The explanatory variable is placed on the x-axis, and relative frequencies (percents) are shown on the y-axis. Each bar reaches 100%.

• Relative Risk: The ratio of the larger percentage to the smaller percentage between groups. It quantifies how many times more likely one group is to display a characteristic compared to another.

• Independence: Two variables are independent if knowing the value of one does not affect the likelihood of the other. In segmented bar graphs, independence is indicated by bars with similar percents.

Example:

Suppose a two-way table shows the frequency of smokers and non-smokers among males and females. Marginal and conditional relative frequencies can be calculated to compare smoking rates by gender. If the segmented bar graph shows similar heights for both genders, smoking and gender may be independent.

HTML Table Example: Two-Way Table Structure

	Category A	Category B	Total
Group 1	Count 1A	Count 1B	Row Total 1
Group 2	Count 2A	Count 2B	Row Total 2
Total	Col Total A	Col Total B	Grand Total

Additional info: Marginal frequencies are found in the row and column totals; conditional frequencies are calculated for each cell relative to its row or column total.

Comparing Two Quantitative Variables

Bivariate quantitative data involves two numeric variables measured for each individual. Scatterplots and correlation coefficients are used to describe and analyze their relationship.

• Scatterplots: Graphs showing paired values for two quantitative variables. The explanatory variable is plotted on the x-axis, and the response variable on the y-axis.

• Describing Scatterplots: Consider strength (strong, moderate, weak), direction (positive or negative), form (linear or non-linear), and unusual features (clusters, outliers).

• Explanatory vs. Response Variable: The explanatory variable is used to predict or explain the response variable.

Example:

A scatterplot of height (x) vs. weight (y) for a group of individuals may show a positive, linear association if taller individuals tend to weigh more.

Correlation

The correlation coefficient, r, measures the strength and direction of the linear relationship between two quantitative variables.

• Properties of r:

  • Always between -1 and 1.

  • r = 1 or r = -1 indicates a perfect linear relationship.

  • r = 0 indicates no linear association.

  • Unit-free; changing units does not affect r.

  • Not resistant; outliers can greatly affect r.

  • Does not distinguish between explanatory and response variables.

  • Measures only linear relationships; does not describe curved associations.

  • Correlation does not imply causation.

Formula for Correlation:

$r = \frac{1}{n-1} \sum \left( \frac{x_i - \bar{x}}{s_x} \right) \left( \frac{y_i - \bar{y}}{s_y} \right)$

Example:

If r = 0.85 for height and weight, there is a strong positive linear association.

Linear Regression Models

Linear regression models predict the response variable using the explanatory variable. The least squares regression line (LSRL) is the best-fitting line that minimizes the sum of squared residuals.

• Regression Equation: $\hat{y} = a + bx$

• Slope (b): Indicates the predicted change in y for each unit increase in x.

• Y-intercept (a): The predicted value of y when x = 0. Sometimes lacks practical meaning.

• Extrapolation: Predicting y for x-values outside the observed range; less reliable.

Formulas:

• Slope: $b = r \frac{s_y}{s_x}$

• Y-intercept: $a = \bar{y} - b \bar{x}$

Example:

If the regression equation for predicting weight from height is $\hat{y} = 50 + 2x$, then for a height of 70 inches, the predicted weight is $\hat{y} = 50 + 2 \times 70 = 190$ pounds.

Residuals and Residual Plots

Residuals measure the difference between observed and predicted values. Residual plots help assess the appropriateness of the regression model.

• Residual: $residual = y - \hat{y}$

• Sum and Mean: The sum and mean of residuals is always zero.

• Residual Plot: Plots residuals against x or predicted y. Random scatter suggests a good fit; patterns indicate model inadequacy.

Example:

If the actual weight is 200 pounds and the predicted weight is 190 pounds, the residual is 10 pounds.

Least Squares Regression Line (LSRL)

The LSRL minimizes the sum of squared residuals and always passes through the point $(\bar{x}, \bar{y})$.

• Equation: $\hat{y} = a + bx$

• Coefficient of Determination ($r^2$): $r^2$ is the proportion of variation in y explained by x via the regression line.

Formula for $r^2$:

$r^2 = (r)^2$

Example:

If $r = 0.8$, then $r^2 = 0.64$, meaning 64% of the variation in y is explained by x.

Departures from Linearity: Outliers, Leverage, and Influence

Unusual features in regression analysis can affect the model's accuracy and interpretation.

• Outlier: A point with a large residual, far from the regression line.

• High Leverage Point: A point with an extreme x-value compared to others.

• Influential Point: A point whose removal significantly changes the regression results.

Example:

A data point with a much higher x-value than others may pull the regression line toward itself, affecting slope and intercept.

Transformations of Data

Transforming variables (e.g., taking logarithms or squares) can help linearize relationships and improve model fit.

• Purpose: To achieve linearity and randomness in residual plots.

• Assessment: Increased randomness in residual plots and higher $r^2$ after transformation suggest a better model.

Example:

Applying a log transformation to y may turn a curved relationship into a linear one, making regression analysis more appropriate.

Additional info: Transformations are commonly used in regression to address non-linearity and heteroscedasticity.