Bayes' Theorem and Law of Total Probability

A 99%-sensitive test on a 1-in-1,000 disease produces a positive result. Intuition says 99% chance of disease. The actual probability is under 2%, because false positives from the 99.9% healthy population overwhelm the true positives.

Let $B_1, \ldots, B_n$ partition $S$ with each $P(B_i) > 0$. For any event $A$:

$$P(A) = \sum_{i=1}^{n} P(A \mid B_i) \cdot P(B_i)$$

The simplest case uses $\{B, B^c\}$:

$$P(A) = P(A \mid B) P(B) + P(A \mid B^c) P(B^c)$$

HIGH-FREQUENCY: The law of total probability appears in nearly every Bayes problem as the denominator. It also appears standalone when a problem gives class-specific conditional probabilities and asks for the overall probability.

Reverses the conditioning direction:

$$P(B_j \mid A) = \frac{P(A \mid B_j) \cdot P(B_j)}{\sum_{i=1}^{n} P(A \mid B_i) \cdot P(B_i)}$$

• $P(B_j)$. prior
• $P(A \mid B_j)$. likelihood
• $P(B_j \mid A)$. posterior
• $P(A)$. marginal likelihood (normalizing constant)

Common mistakes

• Forgetting the denominator. Computing the numerator $P(A \mid B_j) P(B_j)$ correctly but not dividing by $P(A)$. This gives a joint probability, not a conditional. Trap: 0.0225 instead of 0.4206.
• Confusing sensitivity with PPV. Sensitivity is $P(+ \mid D) = 0.95$. PPV is $P(D \mid +) = 0.227$. Reporting 0.95 when asked for PPV is the prosecutor's fallacy. Trap: 0.95.
• Using the wrong false positive rate. Specificity is $P(- \mid D^c)$, so the false positive rate is $1 - \text{specificity}$. Using 0.90 instead of 0.10 for $P(+ \mid D^c)$ gives the wrong denominator. Trap: using specificity directly.

Bottom line

• Law of total probability: $P(A) = \sum P(A \mid B_i) P(B_i)$ over a full partition, and this sum is always the Bayes denominator.
• Bayes' theorem: $P(B_j \mid A) = P(A \mid B_j) P(B_j) / P(A)$, reversing the conditioning direction.
• Terminology: the prior and posterior describe the hypothesis; the likelihood describes the data.
• Posterior odds = likelihood ratio $\times$ prior odds, a faster route than the standard form.

Exam shortcut

Set up a three-column table: (1) scenario, (2) prior times likelihood, (3) posterior. Fill the second column for each $B_i$, sum it to get $P(A)$, then divide each row to get posteriors. "Numerator is the Hit, Denominator is the Law." Numerator = likelihood times prior. Denominator = law of total probability.

The full lesson (about 3,490 words, 23 min read) adds 6 worked examples, all 7 common mistakes, a self-check, free in the app.

Learning objectives

• 1g

Browse all free Exam P lessons or jump into free Exam P practice questions.