Tukey HSD (post-hoc pairwise comparisons)

Compares every pair of group means after an ANOVA while controlling the family-wise error rate, using the studentized range distribution.

SE_{ij} = \sqrt{\frac{MS_W}{2}\left(\frac{1}{n_i} + \frac{1}{n_j}\right)},\qquad q = \frac{|\bar y_i - \bar y_j|}{SE_{ij}}

p-value = P(Q > q) for the studentized range with k groups and N − k df (Tukey–Kramer when sizes differ).

$MS_W$: Within-group mean square from the ANOVA.
$n_i, n_j$: Sizes of the two groups compared.
$q$: Studentized range statistic.

(\bar y_i - \bar y_j) \pm q_{1-\alpha;\,k,\,N-k}\cdot SE_{ij}

Simultaneous confidence intervals: all of them together have (1 − α) coverage.

$q_{1-\alpha;\,k,\,N-k}$: Critical value of the studentized range.

When to use

After a one-way ANOVA (typically a significant one) when you want all pairwise comparisons. For a few pre-planned comparisons, planned contrasts or Holm-adjusted t-tests can be more powerful.

Assumptions

Same as classic ANOVA: independence, approximate normality, equal variances. With unequal sizes the Tukey–Kramer version is slightly conservative.

How to interpret

A pair differs significantly when its adjusted p ≤ α, equivalently when its simultaneous CI excludes 0. The engine reports differences as first − second (R's TukeyHSD prints second − first).

Common mistakes

Using unadjusted t-tests for all pairs (the chance of at least one false positive grows quickly with the number of pairs).
Expecting Tukey results to agree perfectly with the overall F (rarely, F is significant but no pair is, or vice versa).
Using it after Welch's ANOVA when variances clearly differ.

Example

Groups (4, 5, 6, 5), (7, 8, 6, 7), (5, 6, 7, 6): $MS_W = 0.667$ , $n = 4$ , $SE = \sqrt{0.333 \cdot 0.5} = 0.408$ , critical $q = 3.95$ . Group 1 − group 2 $= -2$ : $q = 4.90$ , adjusted $p = 0.018$ , CI −3.61 to −0.39. The other two pairs (difference ±1): $q = 2.45$ , $p = 0.25$ . Only groups 1 and 2 differ convincingly.