Multiple Comparisons: Tukey-Kramer Test: Videos & Practice Problems

After conducting an ANOVA test and rejecting the null hypothesis, the Tukey Kramer test is employed to identify which means differ among groups. This post hoc test compares each pair of means, requiring critical value comparisons using the studentized range distribution table (q table). The q statistic is calculated using the formula: $\frac{(\mathrm{mean}\_1-\mathrm{mean}\_2)}{\sqrt{\left(\mathrm{MSE}\right)(\frac{1}{n\_1}+\frac{1}{n\_2})}}$. A critical value of 3.05 indicates whether to reject the null hypothesis for each pair.

Tukey-Kramer Test

When conducting an ANOVA test, if the null hypothesis is rejected, it indicates that at least one group mean is different from the others. However, it does not specify which means differ, leading to the necessity of post hoc tests. One such test is the Tukey Kramer test, which allows for pairwise comparisons between group means to identify specific differences.

The Tukey Kramer test operates by comparing each possible pair of means. For instance, with three groups, there are three pairs to evaluate. Although this may seem overwhelming, the process involves a few initial steps that are consistent across all comparisons, making it similar to t-tests previously learned.

To begin, ensure that the null hypothesis from the ANOVA test has been rejected, confirming that there is a difference among the means. Set the significance level (alpha) for the Tukey Kramer test, typically at 0.05. The critical value needed for comparisons is obtained from the studentized range distribution table, also known as the q table. This table is similar to the f table used in ANOVA, but it requires the degrees of freedom, calculated as the total number of observations minus the number of groups. For example, with 30 observations across three groups, the degrees of freedom would be 27. Using these values, the critical value can be determined, which in this case is 3.05.

In the Tukey Kramer test, each pair's test statistic, known as the q statistic, is compared against the critical value. If the q statistic exceeds the critical value, the null hypothesis for that pair is rejected, indicating a significant difference between the means. Conversely, if the q statistic is less than the critical value, the null hypothesis is not rejected, suggesting no significant difference.

The q statistic is calculated using the formula:

\[q = \frac{\bar{X}_1 - \bar{X}_2}{\sqrt{\frac{MSE}{n_1} + \frac{MSE}{n_2}}}\]

where \(\bar{X}_1\) and \(\bar{X}_2\) are the means of the two groups being compared, \(MSE\) is the mean squares due to error from the ANOVA output, and \(n_1\) and \(n_2\) are the sample sizes of the respective groups.

For example, when comparing the average study times of grades 10 and 11, if the calculated q statistic is 1.949, which is less than the critical value of 3.05, we fail to reject the null hypothesis, concluding that the average study times are the same. In contrast, when comparing grades 10 and 12, if the q statistic is 4.498, which exceeds the critical value, we reject the null hypothesis, indicating a significant difference in average study times.

By systematically applying the Tukey Kramer test to each pair of means, one can effectively determine which specific means differ, providing clarity following the initial ANOVA analysis.

Tukey-Kramer Test Example 1

In this analysis, we explore the effectiveness of three diet plans (A, B, and C) by employing the Tukey Kramer test following an ANOVA test. The initial step in this process is to confirm that the ANOVA test has been conducted and the null hypothesis has been rejected, indicating that at least one diet plan leads to a different average weight loss. With this established, we proceed to the Tukey Kramer test to identify which specific pairs of means differ.

The first task is to determine the critical value, denoted as q, using the studentized range distribution table. For an alpha level of 0.05, we identify the number of groups (k = 3) and calculate the degrees of freedom as the total number of observations (n = 15) minus the number of groups (3), resulting in 12 degrees of freedom. This yields a critical value of q = 3.773.

Next, we calculate the test statistic for each pair of diet plans. The formula for the test statistic is given by:

q = \frac{|\bar{x}_1 - \bar{x}_2|}{\sqrt{\frac{s^2}{n_1} + \frac{s^2}{n_2}}}

where s² is the mean square error from the ANOVA output, and n represents the sample sizes for each group. In this case, the mean square error is 1.9, and the sample sizes for each diet plan are all 5.

We begin by comparing diet plans A and B. The null hypothesis states that the average weight loss for plan A is equal to that for plan B (μ_A = μ_B), while the alternative hypothesis posits that they are different (μ_A ≠ μ_B). Plugging the means (8 for A and 4.8 for B) into the formula, we calculate:

q = \frac{|8 - 4.8|}{\sqrt{\frac{1.9}{5} + \frac{1.9}{5}}} = 5.19

Since 5.19 exceeds the critical value of 3.773, we reject the null hypothesis, concluding that the average weight loss for plans A and B is significantly different.

Next, we compare plans B and C. The null hypothesis here is μ_B = μ_C, and the alternative is μ_B ≠ μ_C. Using the means (4.8 for B and 11 for C), we find:

q = \frac{|4.8 - 11|}{\sqrt{\frac{1.9}{5} + \frac{1.9}{5}}} = 10.06

Again, since 10.06 is greater than 3.773, we reject the null hypothesis, indicating a significant difference in average weight loss between plans B and C.

Finally, we compare plans A and C. The null hypothesis is μ_A = μ_C, and the alternative is μ_A ≠ μ_C. Using the means (8 for A and 11 for C), we calculate:

q = \frac{|8 - 11|}{\sqrt{\frac{1.9}{5} + \frac{1.9}{5}}} = 4.87

Since 4.87 also exceeds the critical value of 3.773, we reject the null hypothesis, concluding that the average weight loss for plans A and C is significantly different.

In summary, the Tukey Kramer test reveals that all pairs of diet plans (A vs. B, B vs. C, and A vs. C) exhibit significant differences in average weight loss, providing valuable insights into the effectiveness of these diet plans.

Here’s what students ask on this topic:

What is the purpose of the Tukey-Kramer test in statistics?

The Tukey-Kramer test is a post hoc test used after conducting an ANOVA test when the null hypothesis is rejected. Its purpose is to determine which specific group means are significantly different from each other. By comparing all possible pairs of means, the test identifies where the differences lie. It uses a q statistic, derived from the mean squares due to error, and compares it to a critical value from the studentized range distribution table. If the q statistic exceeds the critical value, the means are considered significantly different. This test is particularly useful in experiments with multiple groups, as it controls for Type I error while performing multiple comparisons.

How do you calculate the q statistic in the Tukey-Kramer test?

The q statistic in the Tukey-Kramer test is calculated using the formula:

$q=\frac{|M_1-M_2|}{\sqrt{\frac{{\mathrm{MSE}}_{}}{2}(\frac{1}{n_1}+\frac{1}{n_2})}}$

Here, $M_1$ and $M_2$ are the means of the two groups being compared, ${\mathrm{MSE}}_{}$ is the mean squares due to error from the ANOVA test, and $n_1$ and $n_2$ are the sample sizes of the two groups. The numerator represents the absolute difference between the means, while the denominator accounts for variability and sample size. This q statistic is then compared to a critical value to determine significance.

What is the critical value in the Tukey-Kramer test, and how is it determined?

The critical value in the Tukey-Kramer test is obtained from the studentized range distribution table (q table). It depends on three factors: the significance level (α, typically 0.05), the number of groups being compared, and the degrees of freedom for error. The degrees of freedom for error are calculated as the total number of observations minus the number of groups. Once these values are identified, the critical value is looked up in the q table corresponding to the chosen α level. This critical value serves as the threshold for determining whether the q statistic for a pair of means indicates a significant difference. If the q statistic exceeds the critical value, the null hypothesis for that pair is rejected, indicating a significant difference between the means.

How does the Tukey-Kramer test control for Type I error in multiple comparisons?

The Tukey-Kramer test controls for Type I error by adjusting the significance level across multiple pairwise comparisons. Instead of treating each comparison independently, it uses the studentized range distribution to account for the increased likelihood of false positives when testing multiple pairs. This ensures that the overall probability of making a Type I error (rejecting a true null hypothesis) remains at the chosen significance level, such as 0.05. By comparing the q statistic to a critical value derived from this distribution, the test maintains statistical rigor while identifying significant differences among group means.

What are the assumptions of the Tukey-Kramer test?

The Tukey-Kramer test relies on the following assumptions:

• Normality: The data within each group should be approximately normally distributed.
• Homogeneity of variances: The variances of the groups being compared should be equal. This is often assessed using tests like Levene's test before conducting ANOVA and the Tukey-Kramer test.
• Independent observations: The data points within and between groups should be independent of each other.
• Random sampling: The data should be collected through random sampling to ensure unbiased results.

Violations of these assumptions can affect the validity of the test results, so it is important to check them before proceeding with the analysis.