Home / Courses / Probability / Unit 3

Probability Unit 3: Conditional Probability and Bayes' Theorem

Conditional probability measures how likely an event is once you know another event happened. Bayes' theorem and the law of total probability are among the most tested ideas in any probability course.

Covers: Conditional probability, Multiplication rule and tree diagrams, Independence of events, Law of total probability, Bayes' theorem, Mutual vs pairwise independence, Conditional independence

Study guide

P(A | B) = P(A ∩ B)/P(B) is the probability of A once you know B occurred. Think of it as shrinking the sample space to B and asking what share of B also lies in A. Rearranging gives the multiplication rule P(A ∩ B) = P(B)P(A | B), which is how you compute sequential probabilities such as drawing two aces without replacement: (4/52)(3/51). Tree diagrams organize these calculations: each branch carries a conditional probability, and the probability of a path is the product along it. Conditional probabilities obey all the usual rules, so P(Aᶜ | B) = 1 − P(A | B).

Events are independent when P(A ∩ B) = P(A)P(B), which is the same as P(A | B) = P(A): learning B tells you nothing about A. Independence is a statement about probabilities, not about whether events overlap. Disjoint events with positive probability are always dependent, because learning one occurred makes the other impossible. If A and B are independent, so are their complements. For three or more events, mutual independence needs the product rule for every subcollection, including all three at once; two fair coins with the event 'the coins match' give three events that are pairwise but not mutually independent.

The law of total probability breaks a hard probability into cases. If B₁, …, Bₙ partition the sample space, then P(A) = Σ P(A | Bᵢ)P(Bᵢ). Bayes' theorem then reverses the conditioning: P(Bⱼ | A) = P(A | Bⱼ)P(Bⱼ)/Σ P(A | Bᵢ)P(Bᵢ). In words, the posterior is proportional to prior times likelihood. Medical testing is the classic example. With 1% prevalence, 95% sensitivity and a 5% false-positive rate, only about 16% of positive results are true positives, because the healthy group is so large that its few false positives outnumber the true ones.

Many errors come from confusing P(A | B) with P(B | A), the so-called prosecutor's fallacy. Another subtle idea is conditional independence: two coin flips are independent given which coin was chosen, but if the coin is chosen at random, the first flip carries information about the coin and therefore about the second flip. Sequential Bayesian updating, gambler's ruin and reliability problems all use the same tools: define events carefully, condition on the first step or on the unknown case, and combine with the multiplication and total probability rules.

Key ideas

• Conditioning on B restricts the sample space to B and rescales by P(B).
• The multiplication rule P(A ∩ B) = P(B)P(A | B) drives tree diagrams.
• Independence means P(A ∩ B) = P(A)P(B); disjoint events are usually dependent.
• The law of total probability averages conditional probabilities over a partition.
• Bayes' theorem: posterior ∝ prior × likelihood.
• Low base rates make most positives false even for accurate tests.
• Pairwise independence does not imply mutual independence.

Formulas and rules

Common mistakes

• Confusing P(A | B) with P(B | A), such as treating test sensitivity as the chance of disease.
• Treating mutually exclusive events as independent.
• Multiplying probabilities of dependent events, such as draws without replacement, as if they were independent.
• Ignoring the base rate (prior) when applying Bayes' theorem.
• Checking only pairs when a problem needs mutual independence of three events.

Before the test, make sure you can

• Calculate P(A | B) from a joint probability or by restricting the sample space.
• Use the multiplication rule for draws without replacement.
• Test whether two events are independent using the product rule.
• Apply the law of total probability with a tree diagram.
• Use Bayes' theorem for testing, insurance and urn problems.
• Explain why a rare condition gives a low positive predictive value.
• Distinguish pairwise and mutual independence with an example.
• Compute reliability of series and parallel systems.

Words to know

Conditional probability

P(A | B) = P(A ∩ B)/P(B), the probability of A given that B occurred.

Multiplication rule

P(A ∩ B) = P(B)P(A | B).

Independent events

Events with P(A ∩ B) = P(A)P(B), so knowing one does not change the other's probability.

Mutual independence

Every subcollection of the events satisfies the product rule.

Pairwise independence

Every pair of events is independent; weaker than mutual independence.

Partition

Disjoint events B₁, …, Bₙ whose union is the sample space.

Law of total probability

P(A) = Σ P(A | Bᵢ)P(Bᵢ) over a partition.

Bayes' theorem

P(B | A) = P(A | B)P(B)/P(A).

Prior probability

The probability of a hypothesis before seeing the evidence.

Posterior probability

The updated probability of a hypothesis after seeing the evidence.

Likelihood

The probability of the observed evidence under a given hypothesis.

Sensitivity

P(positive test | condition present).

Specificity

P(negative test | condition absent).

False positive

A positive result for someone without the condition.

Conditional independence

A and B are independent given C if P(A ∩ B | C) = P(A | C)P(B | C).

Tree diagram

A branching picture in which path probabilities are products of conditional probabilities.

Sample practice questions

30 more questions like these, from level 1 to 5, are in the practice set.