Biostatistics Study NotesFrom concepts and derivations to interpretation

CHAPTER 07 · TOPIC 05

Coefficient of Determination (R²)

Interpret the proportion of variation explained by a regression model, its relationship to r², t, and F, and its important limitations.

On this page
  1. Why divide SSR by SST?
  2. Interpreting R² in Plain Language
  3. Scope and Limitations
  4. Why R²=r² in simple regression
  5. Connection between R², t and F
  6. A high R² does not necessarily mean the model is appropriate.

The coefficient of determination, R², represents what proportion of the total variation in Y is explained within the sample by the regression model that includes the intercept. It is an aspect of model fit, not causality, predictive accuracy, or model validity itself.

sum of squares decompositionSST=SSR+SSESS_T=SS_R+SS_E
coefficient of determinationR2=SSRSSTR^2=\frac{SS_R}{SS_T}
equivalence expressionR2=1−SSESSTR^2=1-\frac{SS_E}{SS_T}

Why divide SSR by SST?#

SST is the total variation in Y relative to the mean-only reference model, SSR is the portion explained by the regression, and SSE is the portion remaining. Therefore, R² is the explained part divided by the total, and 1−R² is the proportion of the unexplained part.

Interpreting R² in Plain Language#

100×R2%100\times R^2\%

If R²=0.64, we would say, "In this sample, the model explained 64% of the observed variation in Y." We don't say "X caused 64% of Y" or "The prediction is 64% correct."

Scope and Limitations#

0≤R2≤10\le R^2\le1

If you include the intercept using the ordinary Least Squares and evaluate it on the same data, it will be within this range. Negative values ​​are also possible for predicted R² for no-intercept models, different definitions, and external validation.

Why R²=r² in simple regression#

slopeb1=∑(Xi−Xˉ)(Yi−Yˉ)∑(Xi−Xˉ)2b_1=\frac{\sum(X_i-\bar X)(Y_i-\bar Y)}{\sum(X_i-\bar X)^2}
regression sum of squaresSSR=b12∑(Xi−Xˉ)2SS_R=b_1^2\sum(X_i-\bar X)^2
substitutionR2=[∑(Xi−Xˉ)(Yi−Yˉ)]2∑(Xi−Xˉ)2∑(Yi−Yˉ)2=r2R^2=\frac{[\sum(X_i-\bar X)(Y_i-\bar Y)]^2}{\sum(X_i-\bar X)^2\sum(Y_i-\bar Y)^2}=r^2

Connection between R², t and F#

Simple regression FF=SSR/1SSE/(n−2)F=\frac{SS_R/1}{SS_E/(n-2)}
sum of squaresSSR=R2SST,SSE=(1−R2)SSTSS_R=R^2SS_T,\quad SS_E=(1-R^2)SS_T
F by R²F=R2(n−2)1−R2F=\frac{R^2(n-2)}{1-R^2}
simple regressionF=r2(n−2)1−r2=t2F=\frac{r^2(n-2)}{1-r^2}=t^2

A high R² does not necessarily mean the model is appropriate.#

  • Even if the curve is applied as a straight line, R² can be high if the range is wide.
  • One outlier can push up R²
  • Adding unnecessary explanatory variables usually does not reduce R².
  • Even with high R², unequal variance, dependence, and non-normality of residuals can remain.
  • In-sample R² does not guarantee predictive performance for new data
  • Important small effects may be accurately estimated even with low R²