Bayes' theorem

Bayes' theorem is a simple formula for conditional probabilities ¹. It inverts them: from how often a cause produces an effect, it finds how likely the cause is once you see the effect ². In words, posterior = likelihood × prior ÷ evidence ³. Learning becomes belief revision, where a prior probability is replaced by a posterior that takes in the new information ⁴.

When a disease is rare, the healthy far outnumber the sick, so even a good test can produce more false positives than true ones ¹. The Stanford Encyclopedia's example: a test that catches 95% of heroin users and clears 90% of non-users, where 3% use, leaves a positive tester with a 0.227 chance of being a user ²³; the low base rate swamps the positive result ⁴. Overlooking that is the base rate fallacy ⁵. Compare the false positives of a Bloom filter.

Thomas Bayes never published it. His friend Richard Price edited the manuscript for two years and had it read at the Royal Society on 23 December 1763 ¹; the essay appeared that year, two years after its author's death ². Pierre-Simon Laplace reached and extended the same results in 1774, apparently unaware of Bayes ³.