Home / Courses / Statistics 2 / Unit 8

Statistics 2 Unit 8: Nonparametric Tests

Nonparametric methods replace data values with signs or ranks so that inference does not depend on normality. You will match each rank test to its classical counterpart, compute the test statistics, and interpret results carefully.

Covers: When to use nonparametric methods, The sign test, Wilcoxon signed-rank test, Wilcoxon rank-sum (Mann-Whitney) test, Kruskal-Wallis test, Spearman rank correlation, Permutation and bootstrap methods

Study guide

Classical t and F procedures assume roughly normal populations. With large samples the CLT usually rescues them, but with small samples that are skewed or contain outliers, their p-values and intervals can be badly off. Nonparametric (distribution-free) methods avoid this by working with signs or ranks. An extreme value can only take the largest rank, however extreme it is, so ranks resist outliers. The cost is small: under normality, the Wilcoxon tests are about 95% as efficient as t tests, and with heavy-tailed data they can be much more powerful.

For paired data, the simplest option is the sign test. Drop zero differences, count the plus signs, and compare with Binomial(n, 0.5); the two-sided p-value doubles the smaller tail. Because it ignores the sizes of the differences, the sign test has low power. The Wilcoxon signed-rank test does better: rank the absolute differences from smallest to largest (giving tied values the average rank), then sum the ranks of the positive differences (W⁺) and the negative ones (W⁻). If H₀ of a zero median difference is true, W⁺ and W⁻ should be similar. The test assumes the differences are symmetric about their median.

For two independent samples, the Wilcoxon rank-sum test (equivalently the Mann-Whitney U test) ranks all N observations together and sums the ranks in one group. Under H₀, that sum W has mean n₁(N + 1)/2 and variance n₁n₂(N + 1)/12, which gives a normal approximation for larger samples; U = W − n₁(n₁ + 1)/2. For three or more independent groups, the Kruskal-Wallis test does the same job as one-way ANOVA: H = [12/(N(N + 1))]ΣRᵢ²/nᵢ − 3(N + 1), compared with χ² on k − 1 df. Follow a significant result with adjusted pairwise rank-sum tests.

Interpretation needs care. Rank tests compare distributions: they ask whether values from one group tend to be larger. Only when the groups have the same shape and spread, differing just by a shift, do they become tests of medians. Small samples also limit what rank tests can show, since the number of possible rank arrangements is finite; with five pairs, the signed-rank test can never reach two-sided p < 0.05. For association, Spearman's rank correlation r_s is the Pearson correlation of the ranks, with shortcut 1 − 6Σd²/(n(n² − 1)) when there are no ties. It measures monotonic association and resists outliers.

Modern computing offers resampling methods. A permutation test shuffles group labels many times, recomputing the statistic each time; the p-value is the share of shuffles giving a result at least as extreme as the observed one. It assumes only that labels are exchangeable under H₀, as random assignment guarantees. The bootstrap resamples the data with replacement to approximate the sampling distribution of any statistic, such as a median; the middle 95% of the bootstrap statistics gives a percentile interval. Both approaches let you do inference without formulas tied to normality.

Key ideas

• Rank-based methods do not assume normality and resist outliers.
• The sign test uses only signs; the signed-rank test also uses the sizes of differences through ranks.
• Rank-sum (Mann-Whitney) replaces the two-sample t test; Kruskal-Wallis replaces one-way ANOVA.
• Tied values get the average of the ranks they occupy, and zero differences are dropped.
• Rank tests compare distributions; they test medians only when the shapes match.
• Spearman's correlation measures monotonic association using ranks.
• Permutation tests and the bootstrap use resampling instead of distributional formulas.

Formulas and rules

Common mistakes

• Using the rank-sum test for paired data or the signed-rank test for independent groups.
• Ranking tied values with consecutive ranks instead of their average.
• Keeping zero differences in a sign test or signed-rank test.
• Claiming a rank test shows a difference in medians when the groups have different spreads.
• Forgetting to double a one-tail binomial probability for a two-sided sign test.

Before the test, make sure you can

• Decide when a nonparametric method is preferable to a t or F procedure.
• Carry out a sign test using the binomial distribution.
• Compute W⁺ and W⁻ for a Wilcoxon signed-rank test.
• Compute a rank sum and U for two independent samples, and use the normal approximation.
• Compute the Kruskal-Wallis H statistic and its df.
• Compute and interpret Spearman's rank correlation.
• Explain the logic of permutation tests and bootstrap percentile intervals.
• Interpret rank-test results correctly when group shapes differ.

Words to know

Nonparametric test

A test that does not assume a specific population distribution such as the normal.

Rank

An observation's position when all values are sorted from smallest to largest.

Average rank

The rank given to tied values: the mean of the positions they occupy.

Sign test

A paired test using the count of positive differences, compared with Binomial(n, 0.5).

Wilcoxon signed-rank test

A paired test that ranks absolute differences and sums ranks by sign.

Wilcoxon rank-sum test

A two-independent-sample test based on the sum of one group's ranks.

Mann-Whitney U

An equivalent form of the rank-sum statistic: U = W − n₁(n₁ + 1)/2.

Kruskal-Wallis test

A rank-based test comparing three or more independent groups.

Spearman's rank correlation

The correlation between the ranks of two variables; measures monotonic association.

Efficiency

The relative sample size two tests need for equal power; Wilcoxon vs t is about 0.95 under normality.

Permutation test

A test that builds the null distribution by reshuffling group labels.

Bootstrap

Resampling the data with replacement to approximate a statistic's sampling distribution.

Exchangeability

The assumption that, under H₀, all label assignments are equally likely.

Shift model

The assumption that two distributions have the same shape and differ only in location.

Sample practice questions

30 more questions like these, from level 1 to 5, are in the practice set.