Home / Courses / Statistics 2 / Unit 2

Statistics 2 Unit 2: Two-Sample Inference

Most real questions compare two groups: a treatment and a control, two regions, before and after. This unit shows how to pick the right two-sample method and how to compute and interpret intervals and tests for differences in means and proportions.

Covers: Independent vs paired designs, Two-sample t for means (Welch), Pooled two-sample t, Paired t procedures, Two-proportion z interval and test, Conditions and scope of conclusions

Study guide

The first decision is design. Independent samples come from two separate groups with no link between particular individuals, such as 40 randomly chosen workers from each of two cities. Paired data match each observation in one group with a specific partner in the other: the same person measured before and after, twins split between treatments, or two tires on the same car. The analysis follows the design. For paired data, compute the difference within each pair and run a one-sample t procedure on those differences with n − 1 df, where n is the number of pairs. For independent samples, use a two-sample procedure.

For independent means, the estimate is x̄₁ − x̄₂ and its standard error is √(s₁²/n₁ + s₂²/n₂). Variances add even though means are subtracted, because both sample means contribute random error. The test statistic is t = (x̄₁ − x̄₂)/SE. Welch's version computes df from a formula that falls between min(n₁ − 1, n₂ − 1) and n₁ + n₂ − 2; by hand, the smaller of n₁ − 1 and n₂ − 1 is a conservative choice. The pooled t test instead assumes σ₁ = σ₂, combines the variances into s_p² = [(n₁ − 1)s₁² + (n₂ − 1)s₂²]/(n₁ + n₂ − 2) and uses n₁ + n₂ − 2 df. Welch is the safe default.

Pairing is powerful because it removes variation shared within pairs. If before and after scores are positively correlated, the SD of the differences is √(s₁² + s₂² − 2r·s₁s₂), which can be far smaller than the independent-samples SE. Analyzing paired data as if independent overstates the SE and can hide a real effect; analyzing independent data as if paired is simply invalid because the pairs are arbitrary. Matching helps only when the matching variable is related to the response, since pairs cost degrees of freedom.

For two proportions, the estimate is p̂₁ − p̂₂. A confidence interval uses SE = √(p̂₁(1 − p̂₁)/n₁ + p̂₂(1 − p̂₂)/n₂). A test of H₀: p₁ = p₂ instead pools the data, p̂ = (x₁ + x₂)/(n₁ + n₂), because H₀ says the proportions are equal, and uses SE = √(p̂(1 − p̂)(1/n₁ + 1/n₂)). Because the two SEs differ, the interval and test can disagree slightly in borderline cases. Each group needs at least about 10 successes and 10 failures; with fewer, use an exact method such as Fisher's exact test.

Conditions mirror one-sample inference: random samples or random assignment, independence within and between groups, and approximately normal sampling distributions (check each group's shape when samples are small). Interpretation depends on how the data were gathered. Random sampling lets you generalize to the populations sampled. Random assignment lets you conclude that the treatment caused the difference. An observational comparison of two existing groups, however significant, cannot rule out confounding variables. Always report the estimated difference with its confidence interval, and state the direction of the difference in context.

Key ideas

• The study design, independent or paired, determines the analysis.
• Paired data are analyzed as a one-sample t on the within-pair differences.
• For independent samples, variances add: SE = √(s₁²/n₁ + s₂²/n₂).
• Welch's t does not assume equal variances and is the default; pooled t assumes σ₁ = σ₂.
• Two-proportion tests pool p̂ under H₀; two-proportion intervals do not.
• Pairing gains power only when the matching variable is related to the response.
• Random assignment supports causal claims; random sampling supports generalization.

Formulas and rules

Common mistakes

• Running a two-sample t on before/after data from the same subjects.
• Subtracting variances when computing the SE of a difference.
• Using pooled p̂ in a confidence interval or separate p̂'s in the test.
• Using n₁ + n₂ − 2 df for Welch's test without justification.
• Claiming causation from a significant difference between observational groups.

Before the test, make sure you can

• Identify whether a study produces paired or independent samples.
• Carry out a paired t interval and test on differences.
• Compute the unpooled SE and run Welch's two-sample t test.
• Compute a pooled standard deviation and explain when pooling is justified.
• Construct a two-proportion confidence interval and run a pooled two-proportion z test.
• Check conditions, including large counts for proportions.
• Explain why pairing can increase power and when it does not.
• State whether a conclusion can be causal and to what population it generalizes.

Words to know

Independent samples

Two groups whose individuals are selected or assigned separately with no natural matching.

Paired data

Observations matched one-to-one across groups, such as before/after on the same subject.

Matched pairs design

An experiment that pairs similar units and randomly assigns one of each pair to each treatment.

Paired t test

A one-sample t test on within-pair differences, with n − 1 df for n pairs.

Welch's t test

A two-sample t test that does not assume equal variances and uses an approximate df.

Pooled t test

A two-sample t test assuming equal population variances, with n₁ + n₂ − 2 df.

Pooled variance

s_p², a weighted average of two sample variances using their degrees of freedom as weights.

Pooled proportion

(x₁ + x₂)/(n₁ + n₂), used in the SE of a test of p₁ = p₂.

Two-proportion z test

A test comparing two population proportions using a normal approximation.

Fisher's exact test

An exact test for a 2×2 table, used when counts are too small for z.

Confounding

When another variable is linked to both group membership and the response, muddying causal claims.

Random assignment

Using chance to place units into treatments, which supports cause-and-effect conclusions.

Conservative df

Using min(n₁ − 1, n₂ − 1) for a two-sample t, which slightly overstates p-values.

Difference of means

The parameter μ₁ − μ₂ estimated by x̄₁ − x̄₂.

Sample practice questions

30 more questions like these, from level 1 to 5, are in the practice set.