Biostatistics Study NotesFrom concepts and derivations to interpretation

CHAPTER 03 · TOPIC 03

Hypothesis Testing

Study null and alternative hypotheses, one- and two-sided tests, errors, power, p-values, critical values, and confidence intervals.

On this page
  1. Basic steps for hypothesis testing
  2. Null hypothesis and alternative hypothesis
  3. How the data is processed determines the reference distribution
  4. One-tailed and two-tailed tests
  5. Type I Error, Type II Error, and Statistical Power
  6. What Is a p-Value?
  7. The three judgment methods are looking at the same object
  8. 1. p-value method
  9. 2. Critical value method
  10. 3. Confidence interval method

Hypothesis testing is a method of formulating a testable hypothesis about unknown parameters and determining whether observed results are consistent with that hypothesis based on sample information. Rather than directly proving the truth of a hypothesis, it measures how rare the current result would be if the hypothesis were true.

Basic steps for hypothesis testing#

  • Write the null hypothesis H₀ and alternative hypothesis H₁ (Hₐ) from the research problem
  • Before looking at the results, decide on the significance level α and whether it is one-sided or two-sided.
  • Choose test statistics based on data type, study design, and assumptions
  • Define the sampling distribution of the statistic under H₀
  • Calculate the statistic and p-value from the sample and also compare them with critical values ​​and confidence intervals
  • Decide whether or not to reject H₀, then return to the research question and interpret it.

Null hypothesis and alternative hypothesis#

It transforms a research question into two opposing statistical propositions about the relationship between parameters and populations.

hypothesisroletypical shape
Null hypothesis H₀Starting point for the test. There is no difference/relationship, or the parameter is equal to the specified valueH₀: μ=μ₀
Alternative hypothesis H₁／HₐThe direction we expect the data to supportH₁: μ≠μ₀、μ>μ₀、μ<μ₀
two-sided alternative hypothesisH0:μ=μ0,H1:μ≠μ0H_0:\mu=\mu_0,\qquad H_1:\mu\ne\mu_0
right-sided alternative hypothesisH0:μ≤μ0,H1:μ>μ0H_0:\mu\leq\mu_0,\qquad H_1:\mu>\mu_0
left alternative hypothesisH0:μ≥μ0,H1:μ<μ0H_0:\mu\geq\mu_0,\qquad H_1:\mu<\mu_0

How the data is processed determines the reference distribution#

Different statistics can be created from the same data depending on the question. The selection of Z, t, χ², and F is determined not by the name of the original data, but by which distribution the statistic after the specified operation follows under H₀.

Statistics/research questionsMain reference distribution under H₀
Standardized mean with known population standard deviationStandard normal Z distribution
The mean of the unknown population standard deviation estimated by the sample standard deviationt distribution
Sum of squares of standard normal variables, variance related, some categorical datachi-square distribution
Ratio of two independent variance componentsF distribution

One-tailed and two-tailed tests#

The hem is more extreme than H₀ and supports H₁. A two-tailed test asks "are they different?" and places a rejection region at both ends, while a one-sided test specifies in advance the direction of "greater" or "lesser."

DirectionExample alternative hypothesisRejection region
two-sidedμ≠μ₀Left and right each α/2
right-tailedμ>μ₀Right-tail area α
left-tailedμ<μ₀Left-tail area α

If you have a symmetric Z or t distribution, the direction of observation is consistent with a one-sided H₁, and you are comparing the same statistics, the two-sided p-value is usually twice the corresponding one-sided p-value. It is not applicable to all distributions or observations in the opposite direction.

ptwo-sided=2PH0(T≥∣tobs∣)p_{\mathrm{two\text{-}sided}}=2P_{H_0}(T\geq |t_{\mathrm{obs}}|)

Type I Error, Type II Error, and Statistical Power#

Statistical judgments can be wrong because there is random variation in the sample. Combine and distinguish between true states and judgments.

Judgment/true stateH₀ is trueH₀ is false
Reject H₀Type I error, probability αCorrect judgment, power 1−β
Do not reject H₀Correct judgment, probability 1−αType II error, probability β
Type I errorα=P(reject H0∣H0 is true)\alpha=P(\text{reject }H_0\mid H_0\text{ is true})
Type II errorβ=P(do not reject H0∣H0 is false)\beta=P(\text{do not reject }H_0\mid H_0\text{ is false})
statistical powerPower⁡=1−β\operatorname{Power}=1-\beta

If all other conditions are the same and only the rejection boundary is moved, the lower α is, the easier it is for β to rise. On the other hand, increasing sample size, reducing measurement variation, and increasing the true effect increase power without loosening α.

What Is a p-Value?#

The p-value is the probability of obtaining an extreme result that is as strong or more supportive of the observed test statistic, given that H₀ is true. The "more extreme" direction depends on the predetermined one or both sides.

p=PH0(test statistic at least as extreme as observed)p=P_{H_0}(\text{test statistic at least as extreme as observed})

Determine α (eg 0.05) before analysis, and if p≤α, reject H₀, and if p>α, do not reject. Although a smaller p-value provides stronger evidence against H₀, it does not mean that the effect is large, the study quality is high, or it is clinically important.

Reject H₀p≤αp\leq\alpha
Do not reject H₀p>αp>\alpha

The three judgment methods are looking at the same object#

The p-value method, critical value method, and confidence interval method all return the observed results to the sampling distribution under H₀ and determine whether they exceed a pre-allowed range. These are different perspectives: area, location, and parameter range.

methodWhat to compareConditions for rejecting H₀
p-value methodTail area under H₀ that is more extreme than the observed valuep≤α
critical value methodStatistics and rejection region boundariesExceeds the critical value and enters the rejection region
confidence interval methodIs the population value of H₀ within a reasonable estimation range?In a two-tailed test, the null value corresponds to outside the (1−α) interval

1. p-value method#

From the observed statistics, calculate the tail area that is more extreme than the same in the H₀ distribution and compare it with the predetermined α.

2. Critical value method#

α defines a rejection region of total area α on the H₀ distribution, and the boundary is called the critical value. A statistic crossing the boundary is equivalent to p≤α.

3. Confidence interval method#

A two-tailed test corresponding to the same model and α creates a (1−α)×100% confidence interval. If the specified value of H₀ is outside the interval, it will be rejected; if it is within the interval, it will not be rejected.

H0:θ=θ0,θ0∉CI1−α⟺ptwo-sided<αH_0:\theta=\theta_0,\qquad \theta_0\notin CI_{1-\alpha}\Longleftrightarrow p_{\mathrm{two\text{-}sided}}<\alpha
SymbolMeaning
H₀null hypothesis
H₁、Hₐalternative hypothesis
αSignificance level. Type I error probability when H₀ is true
βType II error probability when the specified H₁ is true
1−βstatistical power
pProbability of obtaining more extreme results than the current one under H₀
CIconfidence interval
critical valueBoundary of rejection region