Pearson Correlation | Regression Analysis | Biostatistics Study Notes

The Pearson product-moment correlation coefficient, r, expresses the direction and strength of a linear relationship between two quantitative variables on a unit-independent scale from -1 to 1.

Calculating Pearson? r#

r=\frac{\sum(X_i-\bar X)(Y_i-\bar Y)}{\sqrt{\sum(X_i-\bar X)^2}\sqrt{\sum(Y_i-\bar Y)^2}}

Expression by covariance

r=\frac{s_{XY}}{s_Xs_Y}

range

-1\le r\le1

A positive value indicates that the variables tend to move in the same direction, whereas a negative value indicates opposite directions. The closer |r| is to 1, the more closely the observations follow a straight line. r=0 only indicates that there is no linear relationship; it does not negate the curvilinear relationship.

Checks before Using Pearson? r#

Both variables are quantitative
The scatterplot shows an approximately linear relationship
Observations are independent
Outliers and mixtures of groups do not dominate r
Bivariate normality or reasonable large sample approximation in inference

Relationship to the Regression Slope#

b_1=r\frac{s_Y}{s_X}

r is scale invariant, but the slope has Y units/X units. Even if the signs are the same, the same r does not necessarily mean the same real effect.

r², R², explained variation#

simple regression

r^2=R^2=\frac{SS_R}{SS_T}

percentage

100r^2\%

In a simple regression with an intercept, r² is the proportion of the total variation in Y that the linear model explains. r itself is not interpreted as a percentage.

Test for population correlation ρ=0#

two-sided hypothesis

H_0:\rho=0,\qquad H_1:\rho\ne0

t-statistic

t=\frac{r\sqrt{n-2}}{\sqrt{1-r^2}}

degrees of freedom

df=n-2

Why correlation tests and regression F tests agree#

relationship to sums of squares

r^2=\frac{SS_R}{SS_T}

t squared

t^2=\frac{r^2(n-2)}{1-r^2}

conversion to F

t^2=\frac{SS_R/1}{SS_E/(n-2)}=F