Home / Courses / Probability / Unit 8

Probability Unit 8: Joint Distributions and Independence

Joint distributions describe two or more random variables together. You will find marginals and conditionals, test independence, and compute probabilities over regions, which is where many exam problems become double integrals.

Covers: Joint PMFs and tables, Joint densities and probabilities over regions, Marginal distributions, Conditional distributions, Independence of random variables, Sums and convolution, Minimum, maximum and order statistics

Study guide

For discrete X and Y, the joint PMF p(x, y) = P(X = x, Y = y) is often given as a table. The marginal of X is found by summing across y, p_X(x) = Σ_y p(x, y), and the conditional PMF of Y given X = x is p(x, y)/p_X(x). For continuous variables a joint density f(x, y) plays the same role: it is nonnegative, has total volume 1, and P((X, Y) ∈ A) = ∬_A f(x, y) dA. Marginals come from integrating out the other variable, f_X(x) = ∫ f(x, y) dy, and conditionals from dividing, f(y | x) = f(x, y)/f_X(x).

Most errors in this unit come from integration limits. Always sketch the support of the joint density and the region of the event. If the support is a triangle such as 0 < y < x < 1, the inner limits depend on the outer variable: for the marginal of Y, x runs from y to 1. A support that is not a rectangle means X and Y are dependent, because knowing one restricts the range of the other. When the density is constant on a region, probabilities are ratios of areas, which can often be found without integrating.

X and Y are independent when the joint distribution factors: p(x, y) = p_X(x)p_Y(y), or f(x, y) = f_X(x)f_Y(y), for every pair. A quick test for densities: the support must be a rectangle and the formula must split into a function of x times a function of y, as in 6e^(−2x − 3y) = 2e^(−2x) × 3e^(−3y). One table cell that fails the product rule proves dependence. Under independence, probabilities of 'X in A and Y in B' events multiply, and so do survival functions.

Functions of several variables are found the same way as in one dimension. For a sum of independent variables, the density is the convolution f_Z(z) = ∫ f_X(x) f_Y(z − x) dx; two uniforms on [0, 1] give a triangular density, and two exponentials give a gamma. For the maximum and minimum of independent variables, P(max ≤ t) = Π Fᵢ(t) and P(min > t) = Π Sᵢ(t). The k-th smallest of n values, an order statistic, is at most t exactly when at least k of the values are at most t, which is a binomial count.

Key ideas

• Marginals come from summing or integrating out the other variable.
• Conditionals are joint divided by marginal and must sum or integrate to 1.
• Sketch the support first; triangular supports change the limits of integration.
• Independence requires a rectangular support and a factored density.
• Probabilities over regions are double integrals or, for uniform densities, area ratios.
• The density of a sum of independent variables is a convolution.
• For independent variables, CDFs multiply for the max and survival functions for the min.

Formulas and rules

Common mistakes

• Using constant limits when the support is a triangle or another non-rectangular region.
• Forgetting to divide by the marginal when forming a conditional distribution.
• Declaring independence because a formula factors while the support is not a rectangle.
• Treating area as probability when the joint density is not constant.
• Writing a marginal that still depends on the other variable.

Before the test, make sure you can

• Read a joint PMF table to find marginals and conditionals.
• Find the constant that makes a joint density valid.
• Set up double integrals with correct limits for triangular regions.
• Derive marginal and conditional densities and conditional expectations.
• Test independence with the product rule or the factorization test.
• Find distributions of the maximum and minimum of independent variables.
• Use convolution or known results for sums of independent variables.
• Compute order statistic probabilities with a binomial count.

Words to know

Joint PMF

p(x, y) = P(X = x, Y = y) for discrete X and Y.

Joint density

f(x, y) ≥ 0 with total volume 1; probabilities are volumes over regions.

Marginal distribution

The distribution of one variable alone, found by summing or integrating out the others.

Conditional distribution

The distribution of Y given X = x: f(x, y)/f_X(x).

Independent random variables

Variables whose joint distribution equals the product of their marginals.

Support

The region where the joint density is positive.

Convolution

The integral ∫ f_X(x) f_Y(z − x) dx giving the density of an independent sum.

Order statistics

The sorted values X₍₁₎ ≤ X₍₂₎ ≤ ··· ≤ X₍ₙ₎ of a sample.

Distribution of the maximum

For independent variables, P(max ≤ t) = Π P(Xᵢ ≤ t).

Distribution of the minimum

For independent variables, P(min > t) = Π P(Xᵢ > t).

Triangular distribution

The density of the sum of two independent uniform(0, 1) variables, peaking at 1.

Bivariate uniform

A joint density constant on a region, so probabilities are ratios of areas.

Factorization test

A joint density on a rectangle that splits as g(x)h(y) means independence.

Sample practice questions

30 more questions like these, from level 1 to 5, are in the practice set.