Fisher’s Exact Test | Binomial Tests and 2×2 Tables

Fisher's exact test calculates the probability of each table meeting the null hypothesis directly from the hypergeometric distribution, under the condition that the marginal frequencies of the rows and columns of the 2 × 2 table are fixed. It does not rely on large-sample approximations, so it is important for small samples or when expected frequencies are small.

With Fixed Marginal Totals, Only One Cell Is Free#

marginal total

R_1=a+b,\quad R_2=c+d,\quad C_1=a+c,\quad C_2=b+d

Determine other cells from a

b=R_1-a,\quad c=C_1-a,\quad d=R_2-C_1+a

minimum value of a

a_{\min}=\max(0,R_1-C_2)

maximum value of a

a_{\max}=\min(R_1,C_1)

Exact Probability of the Observed Table#

hypergeometric probability

P(A=a\mid R_1,R_2,C_1,C_2)=\frac{\binom{C_1}{a}\binom{C_2}{R_1-a}}{\binom{N}{R_1}}

factorial display

P(a,b,c,d)=\frac{R_1!R_2!C_1!C_2!}{a!b!c!d!N!}

The denominator is how to choose R₁ items that go into the first row out of N items. The combination of selecting a pieces from the first column and R₁−a pieces from the second column becomes the numerator, and the probability of a table that maintains the marginal frequency is obtained.

\frac{\binom{C_1}{a}\binom{C_2}{b}}{\binom{N}{R_1}}=\frac{R_1!R_2!C_1!C_2!}{a!b!c!d!N!}

One-Sided p-Value#

right side

p_{\mathrm{right}}=\sum_{a\geq a_{\mathrm{obs}}}P(A=a\mid\text{margins})

left

p_{\mathrm{left}}=\sum_{a\leq a_{\mathrm{obs}}}P(A=a\mid\text{margins})

Which tail supports the alternative hypothesis is determined by the row/column arrangement and the predetermined direction of the effect. Don't look at the data and choose the smaller one-sided p-value.

Two-Sided p-Value#

p_{\mathrm{two\text{-}sided}}=\sum_{T:\,P(T)\leq P(T_{\mathrm{obs}})}P(T)

Steps to enumerate all tables#

Calculate R₁, R₂, C₁, C₂, N from the observation table
Enumerate possible a from a_min to a_max
Calculate b, c, d and hypergeometric probability for each a
Define one-sided or two-sided extreme tables depending on alternative hypothesis
Sum the corresponding probabilities to get the p-value

Comparison of Pearson, Yates, and Fisher#

method	statistical foundations	approximation	Typical use
Pearson χ²	Σ(O−E)²/E	chi-square approximation	sufficient expectation
Yates χ²	Σ(\|O−E\|−0.5)²/E	Corrected chi-square approximation	Smaller 2×2 table, conservative reasoning
Fisher accurate	conditional hypergeometric distribution	Unnecessary	Small sample/sparse 2×2 table

Even though the p-value of the Fisher test is accurate, it does not mean that there is no uncertainty in the effect estimate. Report the OR, appropriate confidence interval, and cell frequency, understanding that it is a conditional inference.