Statistics Quest

Fisher's exact test

An exact test of association in a 2 × 2 table that conditions on both margins, valid for any sample size.

P(a)=(a+ba)(c+dc)(Na+c)P(a) = \frac{\binom{a+b}{a}\binom{c+d}{c}}{\binom{N}{a+c}}
Probability of the table [[a, b], [c, d]] given its margins (hypergeometric), under H₀ of no association.
a,b,c,da, b, c, d
Cell counts (rows = groups, columns = outcome).
NN
a+b+c+da + b + c + d.
ptwo-sided=∑x: P(x)≤P(a)P(x)p_{\text{two-sided}} = \sum_{x:\ P(x) \le P(a)} P(x)
Two-sided p: sum of the probabilities of all tables (same margins) no more likely than the observed one (R's rule). One-sided: P(X ≤ a) or P(X ≥ a).
xx
Possible values of the top-left cell given the margins.

When to use

2×2 tables with small expected counts (any below 5), or whenever an exact p-value is preferred: rare adverse events, small pilot studies.

Assumptions

Independent observations; the margins are treated as fixed. Because of the discreteness, the test is conservative (true Type I error rate ≤ α).

How to interpret

The engine reports two odds ratios: the sample OR ad/(bc)ad/(bc) and the conditional maximum-likelihood OR with its exact CI (as R). A CI excluding 1 matches a two-sided p below α.

Common mistakes

  • Doubling a one-sided p-value instead of using the two-sided rule.
  • Reading the very wide CI from a small table as precise evidence of a huge effect.
  • Using it on larger r×c tables without an engine that supports them (this engine: 2×2 only).

Example

Treated: 7 improved, 1 not; control: 2 improved, 6 not. Two-sided p=0.041p = 0.041. Sample OR =(7⋅6)/(1⋅2)=21= (7\cdot6)/(1\cdot2) = 21; conditional MLE OR 16.2 with 95% CI 1.09 to 1099 — evidence of an association, but its size is extremely uncertain with 16 subjects.