Analysis of Variance (ANOVA): One-Way and Two-Way Methods

Analysis of Variance (ANOVA)

Introduction

Analysis of Variance (ANOVA) is a statistical method used to compare means across multiple groups to determine if at least one group mean is significantly different from the others. ANOVA is widely used in business statistics to analyze experimental data and to test hypotheses about group differences.

General ANOVA Setting

Experimental Design and Factors

• Factor: A variable controlled by the investigator, which can have two or more levels (categories or numerical values).

• Levels: Different values or categories of a factor, each producing a distinct group.

• Each group is considered a sample from a different population.

• Dependent Variable: The outcome measured to observe the effect of the factor(s).

• Experimental Design: The plan used to collect and assign data to groups.

Completely Randomized Design

One-Way ANOVA

• Subjects (experimental units) are randomly assigned to groups, assuming homogeneity.

• Only one factor (independent variable) is considered, with two or more levels.

• Analysis is performed using one-way ANOVA.

One-Way Analysis of Variance

Purpose and Assumptions

• Evaluates differences among the means of three or more groups.

• Assumptions:

  • Populations are normally distributed.

  • Populations have equal variances.

  • Samples are randomly and independently selected.

• Example: Comparing the number of accidents for different work shifts or expected mileage for different brands of tires.

Hypotheses of One-Way ANOVA

Formulation

• Null Hypothesis ($H_0$): All population means are equal. $H_0: \mu_1 = \mu_2 = \mu_3 = \cdots = \mu_c$

• Alternative Hypothesis ($H_1$): At least one population mean is different. $H_1:$ Not all of the population means are equal

• Rejecting $H_0$ indicates a factor effect, but does not specify which means differ.

Partitioning the Variation

Components of Variation

• Total variation in the data is split into two parts:

Formula: $SST = SSA + SSW$

• SST (Total Sum of Squares): Aggregate variation of all data values.

• SSA (Sum of Squares Among Groups): Variation among group means.

• SSW (Sum of Squares Within Groups): Variation within each group.

Calculating Sums of Squares

Total Sum of Squares

Formula: $SST = \sum_{j=1}^{c} \sum_{i=1}^{n_j} (X_{ij} - \overline{X})^2$

• $c$ = number of groups

• $n_j$ = number of values in group $j$

• $X_{ij}$ = $i$th observation from group $j$

• $\overline{X}$ = grand mean (mean of all data values)

Among-Group Variation

Formula: $SSA = \sum_{j=1}^{c} n_j (\overline{X}_j - \overline{X})^2$

• $\overline{X}_j$ = mean of group $j$

Mean Square Among: $MSA = \frac{SSA}{c-1}$

Within-Group Variation

Formula: $SSW = \sum_{j=1}^{c} \sum_{i=1}^{n_j} (X_{ij} - \overline{X}_j)^2$

Mean Square Within: $MSW = \frac{SSW}{n-c}$

Obtaining the Mean Squares

• Mean Square Among (MSA): $MSA = \frac{SSA}{c-1}$

• Mean Square Within (MSW): $MSW = \frac{SSW}{n-c}$

• Mean Square Total (MST): $MST = \frac{SST}{n-1}$

One-Way ANOVA Table

Summary Table

Source of Variation	Degrees of Freedom	Sum Of Squares	Mean Square (Variance)	F
Among Groups	c - 1	SSA	$MSA = \frac{SSA}{c-1}$	$F_{STAT} = \frac{MSA}{MSW}$
Within Groups	n - c	SSW	$MSW = \frac{SSW}{n-c}$
Total	n - 1	SST

One-Way ANOVA F Test Statistic

Test Statistic and Degrees of Freedom

• Test Statistic: $F_{STAT} = \frac{MSA}{MSW}$

• Degrees of Freedom:

  • $df_1 = c - 1$ (numerator, among groups)

  • $df_2 = n - c$ (denominator, within groups)

Interpreting the F Statistic

• The F statistic is the ratio of the among-group variance estimate to the within-group variance estimate.

• The ratio must always be positive.

• Decision Rule: Reject $H_0$ if $F_{STAT} > F_{\alpha}$ (critical value from F-distribution table).

One-Way ANOVA Example

Golf Club Distance Example

• Three golf clubs tested for mean distance using five measurements each.

• Data:

Club 1	Club 2	Club 3
254	234	200
263	218	222
241	235	197
237	227	206
251	216	204

• Means: $\overline{X}_1 = 249.2$, $\overline{X}_2 = 226.0$, $\overline{X}_3 = 205.8$, $\overline{X} = 227.0$

• SSA, SSW, MSA, MSW, and $F_{STAT}$ are calculated as shown in the slides.

• Decision: Since $F_{STAT} = 25.275 > F_{\alpha} = 3.89$, reject $H_0$ at $\alpha = 0.05$.

• Conclusion: There is evidence that at least one mean differs from the rest.

Assumptions for ANOVA F Test

• Randomness and Independence: Samples must be randomly and independently selected.

• Normality: Populations should be normally distributed. The F test is robust to moderate departures from normality, especially with large samples.

• Homogeneity of Variance: Group variances should be equal. This can be tested using Levene's Test.

Post-Hoc Analysis: Tukey-Kramer Procedure

Purpose and Steps

• Used after rejecting $H_0$ in ANOVA to determine which means are significantly different.

• Allows paired comparisons by comparing absolute mean differences to a critical range.

• Critical Range Formula: $\text{Critical Range} = Q_{\alpha} \sqrt{\frac{MSW}{2} \left( \frac{1}{n_1} + \frac{1}{n_2} \right)}$ Additional info: The formula may vary slightly depending on sample sizes and the number of groups.

• If the absolute mean difference exceeds the critical range, the means are significantly different.

Two-Way ANOVA

Introduction and Assumptions

• Used when there are two factors of interest, each with multiple levels.

• Assumptions:

  • Populations are normally distributed.

  • Populations have equal variances.

  • Independent random samples are selected.

Sources of Variation in Two-Way ANOVA

• Factor A Variation (SSA): Variation due to levels of factor A.

• Factor B Variation (SSB): Variation due to levels of factor B.

• Interaction Variation (SSAB): Variation due to interaction between factors A and B.

• Error Variation (SSE): Random variation within cells.

• Total Variation (SST): Sum of all sources of variation.

Two-Way ANOVA Equations

• Total Variation: $SST = \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n'} (X_{ijk} - \overline{X})^2$

• Factor A Variation: $SSA = b n' \sum_{i=1}^{a} (\overline{X}_{i..} - \overline{X})^2$

• Factor B Variation: $SSB = a n' \sum_{j=1}^{b} (\overline{X}_{.j.} - \overline{X})^2$

• Interaction Variation: $SSAB = n' \sum_{i=1}^{a} \sum_{j=1}^{b} (\overline{X}_{ij.} - \overline{X}_{i..} - \overline{X}_{.j.} + \overline{X})^2$

• Error Variation: $SSE = \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n'} (X_{ijk} - \overline{X}_{ij.})^2$

Mean Square Calculations

• $MSA = \frac{SSA}{a-1}$

• $MSB = \frac{SSB}{b-1}$

• $MSAB = \frac{SSAB}{(a-1)(b-1)}$

• $MSE = \frac{SSE}{ab(n'-1)}$

Interpreting Two-Way ANOVA Results

• First, test for statistical significance of the interaction effect.

• If interaction is significant, focus analysis on the interaction.

• If interaction is not significant, focus on main effects (Factor A and Factor B).

Two-Way ANOVA Summary Table

Source of Variation	Degrees of Freedom	Sum of Squares	Mean Square	F Statistic
Factor A	a - 1	SSA	$MSA = \frac{SSA}{a-1}$	$F_A = \frac{MSA}{MSE}$
Factor B	b - 1	SSB	$MSB = \frac{SSB}{b-1}$	$F_B = \frac{MSB}{MSE}$
Interaction	(a-1)(b-1)	SSAB	$MSAB = \frac{SSAB}{(a-1)(b-1)}$	$F_{AB} = \frac{MSAB}{MSE}$
Error	ab(n'-1)	SSE	$MSE = \frac{SSE}{ab(n'-1)}$
Total	abn' - 1	SST

Key Features and Summary

• ANOVA is essential for comparing means across multiple groups in business statistics.

• One-way ANOVA tests for differences among group means for a single factor; two-way ANOVA extends this to two factors and their interaction.

• Assumptions of normality, equal variances, and random sampling are critical for valid inference.

• Post-hoc tests like Tukey-Kramer help identify which means differ after a significant ANOVA result.