Intuitive explanation of Bayes' theorem with real-world examples

Bayes' Theorem is how we **update beliefs** when we see new evidence. **Why it feels backwards:** $P(A|B)$ is the probability of $A$ **after** seeing $B$. $P(A)$ is the probability **before** seeing $B$. Bayes tells us how evidence $B$ should change our belief in $A$. We often know the \"likelihood\" $P(B|A)$ from data, and the \"prior\" $P(A)$ from background knowledge, and we want to compute the \"posterior\" $P(A|B)$. **Medical Testing Example:** A disease affects $1\%$ of the population: $P(D) = 0.01$. A test is $99\%$ accurate: $P(\text{positive}|D) = 0.99$, $P(\text{negative}|\ eg D) = 0.99$. If you test positive, what is $P(D|\text{positive})$? $$P(D|+) = \frac{P(+|D)P(D)}{P(+)} = \frac{0.99 \times 0.01}{0.99 \times 0.01 + 0.01 \times 0.99} = \frac{0.0099}{0.0198} = 0.5$$ Surprisingly, only $50\%$! Even with a $99\%$ accurate test, the low base rate means a positive result is not definitive. **Spam Filtering:** Given an email containing the word \"free\", what is $P(\text{spam}|\text{free})$? We know: - $P(\text{spam}) = 0.6$ (60% of emails are spam) - $P(\text{free}|\text{spam}) = 0.8$ (80% of spam contains \"free\") - $P(\text{free}|\text{not spam}) = 0.1$ (10% of legitimate emails contain \"free\") $$P(\text{spam}|\text{free}) = \frac{0.8 \times 0.6}{0.8 \times 0.6 + 0.1 \times 0.4} = \frac{0.48}{0.52} \approx 0.923$$ Bayes' Theorem is the mathematical foundation of all rational learning from data.