CHAPTER 08 · TOPIC 04
Appendix: Selecting a Statistical Test
Choose a statistical method from the outcome type, sample relationship, target hypothesis, and assumptions of the candidate test.
On this page
Statistical methods are selected in the following order: the type of outcome variable, the relationship between the samples, and the population-level hypotheses to be compared. The table below is a starting point, and the final decision is made by reviewing the study design and assumptions for each method.
Integrated Statistical-Test Selection Table#
| Outcome | One sample | Two independent groups | Two paired time points | Three or more independent groups |
| Binary | Binomial | χ² / Fisher | McNemar | χ² |
| Approximately normal continuous | One-sample t | Independent t / Welch t | Paired t | ANOVA / Welch ANOVA |
| Skewed continuous or outlier-prone | Sign / Signed-rank | Rank-sum | Sign / Signed-rank | Kruskal–Wallis |
| Ordinal | Sign / Signed-rank | Rank-sum | Sign / Signed-rank | Kruskal–Wallis |
Question 1: What Type of Outcome Variable?#
| Type | Examples | Primary comparison |
| Binary | Present/absent, positive/negative | Proportion or event probability |
| Continuous | Blood pressure, weight | Mean, location, or distribution |
| Ordinal | Disease stage, pain scale | Ranks or distributional location |
Coding categories as 0, 1, and 2 does not make them continuous measurements. Likewise, a significant normality test alone does not automatically require switching to a nonparametric method.
Question 2: How Are the Samples Related?#
| Relationship | Definition | Example |
| One sample | One group compared with a reference value | Is the median 4? |
| Independent | Different participants belong to different groups | Treatment and control groups |
| Paired | Repeated or matched observations | Before and after treatment |
Question 3: What Does H₀ Compare?#
| Method | Primary null hypothesis |
| t / ANOVA | The specified means are equal |
| Sign | Positive and negative differences are equally likely |
| Signed-rank | The difference distribution is symmetric around zero |
| Rank-sum | The two independent groups have the same distribution |
| Median | The proportions on both sides of the pooled median are equal |
| Kruskal–Wallis | All independent groups have the same distribution |
| χ² independence | Two categorical variables are independent |
Methods described broadly as a "two-group comparison" can test different hypotheses. A rank test can be interpreted as comparing medians only under additional conditions, such as similarly shaped distributions that differ solely by a location shift.
Checks after Using the Table#
- See outliers, skewness, variance, and clusters in plots
- Write your research question and H₀ and check if your method answers the same question
- Check independence, pairing, expected counts, symmetry, homoscedasticity, ties
- Specify exact, permutation, and asymptotic methods and corrections for small samples.
- Report effect sizes and confidence intervals and do not reduce the analysis to p values alone