Biostatistics Study NotesFrom concepts and derivations to interpretation

CHAPTER 06 · TOPIC 01

One-Way ANOVA

Treat group membership as a model, decompose variation into between- and within-group components, and construct the F test.

On this page
  1. The core of ANOVA: treating grouping as a model
  2. Data Structure and Assumptions
  3. Why can we merge within-group sums of squares?
  4. Why does the ratio of squares have an F distribution?
  5. Hypotheses
  6. Symbols and one-way models
  7. Decomposing the Sum of Squares
  8. Degrees of Freedom, Mean Squares, and the F Test
  9. Complete Analysis Procedure
  10. With Two Groups, F=t²
  11. Extensions of ANOVA

One-way ANOVA compares the population means of three or more independent groups at once. The total variation is decomposed into between-group variation and within-group variation, and the ratio determines whether the group effect is sufficiently larger than chance variation.

The core of ANOVA: treating grouping as a model#

F-statisticF=MSbetweenMSwithin=MSMMSEF=\frac{MS_{\mathrm{between}}}{MS_{\mathrm{within}}}=\frac{MS_M}{MS_E}
Distribution under H₀F∼Fk−1, n−kF\sim F_{k-1,\,n-k}

Data Structure and Assumptions#

itemrequirements
explanatory variablesOne categorical factor (k levels)
dependent variableQuantitative variables whose averages have meaning
sampleEach object belongs to only one independent group
null hypothesisThe population means of all groups are equal
  • observation units are independent of each other
  • Errors in each group are approximately normally distributed
  • Each group has a common error variance σ²
  • The dependent variable is quantitative and mean comparison is meaningful for research.

Why can we merge within-group sums of squares?#

Within-group sum of squares of group jSSE,j=∑i=1nj(Xij−Xˉj)2SS_{E,j}=\sum_{i=1}^{n_j}(X_{ij}-\bar X_j)^2
Merge all groupsSSE=∑j=1k∑i=1nj(Xij−Xˉj)2SS_E=\sum_{j=1}^{k}\sum_{i=1}^{n_j}(X_{ij}-\bar X_j)^2
Estimation of common within-group varianceMSE=SSEn−kMS_E=\frac{SS_E}{n-k}
Within-group degrees of freedom∑j=1k(nj−1)=n−k\sum_{j=1}^{k}(n_j-1)=n-k

If each group estimates the same error variance, the independent error information for each group can be weighted and merged by degrees of freedom. This result is MSE.

Why does the ratio of squares have an F distribution?#

between groupsSSMσ2∼χk−12\frac{SS_M}{\sigma^2}\sim\chi^2_{k-1}
within the groupSSEσ2∼χn−k2\frac{SS_E}{\sigma^2}\sim\chi^2_{n-k}
(SSM/σ2)/(k−1)(SSE/σ2)/(n−k)=MSMMSE∼Fk−1, n−k\frac{(SS_M/\sigma^2)/(k-1)}{(SS_E/\sigma^2)/(n-k)}=\frac{MS_M}{MS_E}\sim F_{k-1,\,n-k}

Under H₀ both equations estimate σ², so the ratio is roughly around 1. If the group means differ, MSM increases and F increases to the right.

Hypotheses#

null hypothesisH0:μ1=μ2=⋯=μkH_0:\mu_1=\mu_2=\cdots=\mu_k
alternative hypothesisH1:Not all μj are equalH_1:\text{Not all }\mu_j\text{ are equal}
equivalent expressionH1:∃ j≠ℓ such that μj≠μℓH_1:\exists\,j\ne\ell\text{ such that }\mu_j\ne\mu_\ell

Symbols and one-way models#

jth group averageXˉj=1nj∑i=1njXij\bar X_j=\frac{1}{n_j}\sum_{i=1}^{n_j}X_{ij}
grand averageXˉ⋅⋅=1n∑j=1knjXˉj\bar X_{\cdot\cdot}=\frac{1}{n}\sum_{j=1}^{k}n_j\bar X_j
population modelXij=μ+αj+εijX_{ij}=\mu+\alpha_j+\varepsilon_{ij}
errorεij∼iidN(0,σ2)\varepsilon_{ij}\overset{\mathrm{iid}}{\sim}N(0,\sigma^2)
Xij−Xˉ⋅⋅⏟total deviation=Xˉj−Xˉ⋅⋅⏟between groups+Xij−Xˉj⏟within group\underbrace{X_{ij}-\bar X_{\cdot\cdot}}_{\text{total deviation}}=\underbrace{\bar X_j-\bar X_{\cdot\cdot}}_{\text{between groups}}+\underbrace{X_{ij}-\bar X_j}_{\text{within group}}

Decomposing the Sum of Squares#

within-group sum of squaresSSE=∑j∑i(Xij−Xˉj)2SS_E=\sum_j\sum_i(X_{ij}-\bar X_j)^2
between-group sum of squaresSSM=∑jnj(Xˉj−Xˉ⋅⋅)2SS_M=\sum_j n_j(\bar X_j-\bar X_{\cdot\cdot})^2
total sum of squaresSST=∑j∑i(Xij−Xˉ⋅⋅)2SS_T=\sum_j\sum_i(X_{ij}-\bar X_{\cdot\cdot})^2
DecompositionSST=SSM+SSESS_T=SS_M+SS_E

Degrees of Freedom, Mean Squares, and the F Test#

source of variationsum of squaresdegree of freedomUniform square
between groupsSSMk−1SSM/(k−1)
within the groupSSEn−kSSE/(n−k)
wholeSSTn−1—
F-statisticFobs=MSMMSEF_{\mathrm{obs}}=\frac{MS_M}{MS_E}
right tail p-valuep=P ⁣(Fk−1, n−k≥Fobs∣H0)p=P\!\left(F_{k-1,\,n-k}\ge F_{\mathrm{obs}}\mid H_0\right)
degree of freedom decompositionn−1=(k−1)+(n−k)n-1=(k-1)+(n-k)

Complete Analysis Procedure#

  1. Confirm study design, independence, factor levels, and dependent variables
  2. Check n, mean, standard deviation, distribution, and outliers by group
  3. Diagnose residuals and homoscedasticity and select Welch method if necessary
  4. Calculate sum of squares, degrees of freedom, mean squares, F, p-value
  5. If significant, perform planned contrasts or post hoc comparisons adjusted for multiplicity.
  6. Report mean differences, confidence intervals, effect sizes, assumptions and limitations

With Two Groups, F=t²#

With two independent groups assuming equal variances, one-way ANOVA and pooled t-tests test the same null hypothesis.

Pooled tt=Xˉ1−Xˉ2sp2(1/n1+1/n2)t=\frac{\bar X_1-\bar X_2}{\sqrt{s_p^2(1/n_1+1/n_2)}}
pooled variancesp2=(n1−1)s12+(n2−1)s22n1+n2−2s_p^2=\frac{(n_1-1)s_1^2+(n_2-1)s_2^2}{n_1+n_2-2}
MSE of ANOVAMSE=sp2MS_E=s_p^2
Between-group equationMSM=(Xˉ1−Xˉ2)21/n1+1/n2MS_M=\frac{(\bar X_1-\bar X_2)^2}{1/n_1+1/n_2}
conclusionF=t2\boxed{F=t^2}

Extensions of ANOVA#

Consider Welch ANOVA for unequal variances, two-way ANOVA that handles main effects and interactions for two factors, MANOVA for multiple quantitative dependent variables, and repeated measures or mixed models for repeated measurements.