Bayes' Theorem and Law of Total Probability | Free Exam P Lesson

A 99%-sensitive test on a 1-in-1,000 disease produces a positive result. Intuition says 99% chance of disease. The actual probability is under 2%, because false positives from the 99.9% healthy population overwhelm the true positives.

Let $B_{1}, \dots, B_{n}$ partition $S$ with each $P (B_{i}) > 0$ . For any event $A$ :

$P (A) = i = 1 \sum n P (A ∣ B_{i}) \cdot P (B_{i})$

The simplest case uses ${B, B^{c}}$ :

$P (A) = P (A ∣ B) P (B) + P (A ∣ B^{c}) P (B^{c})$

HIGH-FREQUENCY: The law of total probability appears in nearly every Bayes problem as the denominator. It also appears standalone when a problem gives class-specific conditional probabilities and asks for the overall probability.

Reverses the conditioning direction:

$P (B_{j} ∣ A) = \frac{P ( A ∣ B _{j} ) \cdot P ( B _{j} )}{\sum _{i = 1}^{n} P ( A ∣ B _{i} ) \cdot P ( B _{i} )}$

$P (B_{j})$ . prior
$P (A ∣ B_{j})$ . likelihood
$P (B_{j} ∣ A)$ . posterior
$P (A)$ . marginal likelihood (normalizing constant)

Read the full lesson, free →

Worked examples and practice. Free with a free account, no card.

Common mistakes

Forgetting the denominator. Computing the numerator $P (A ∣ B_{j}) P (B_{j})$ correctly but not dividing by $P (A)$ . This gives a joint probability, not a conditional. Trap: 0.0225 instead of 0.4206.
Confusing sensitivity with PPV. Sensitivity is $P (+ ∣ D) = 0.95$ . PPV is $P (D ∣ +) = 0.227$ . Reporting 0.95 when asked for PPV is the prosecutor's fallacy. Trap: 0.95.
Using the wrong false positive rate. Specificity is $P (- ∣ D^{c})$ , so the false positive rate is $1 - specificity$ . Using 0.90 instead of 0.10 for $P (+ ∣ D^{c})$ gives the wrong denominator. Trap: using specificity directly.

Bottom line

Law of total probability: $P (A) = \sum P (A ∣ B_{i}) P (B_{i})$ over a full partition, and this sum is always the Bayes denominator.
Bayes' theorem: $P (B_{j} ∣ A) = P (A ∣ B_{j}) P (B_{j}) / P (A)$ , reversing the conditioning direction.
Terminology: the prior and posterior describe the hypothesis; the likelihood describes the data.
Posterior odds = likelihood ratio $\times$ prior odds, a faster route than the standard form.

Exam shortcut

Set up a three-column table: (1) scenario, (2) prior times likelihood, (3) posterior. Fill the second column for each $B_{i}$ , sum it to get $P (A)$ , then divide each row to get posteriors. "Numerator is the Hit, Denominator is the Law." Numerator = likelihood times prior. Denominator = law of total probability.

The full lesson (about 3,490 words, 23 min read) adds 6 worked examples, all 7 common mistakes, a self-check, free in the app.

Learning objectives

Browse all free Exam P lessons or jump into free Exam P practice questions.