Bayes' rule (aka Bayes' theorem) is the quantitative law of probability theory governing how to revise probabilistic beliefs in response to observing new evidence.

You may want to start at the Guide or the Fast Intro.

The laws of reasoning

Imagine that, as part of a clinical study, you're being tested for a rare form of cancer, which affects 1 in 10,000 people. You have no reason to believe that you are more or less likely than average to have this form of cancer. You're administered a test which is 99% accurate, both in terms of specificity and sensitivity: It correctly detects the cancer (in patients who have it) 99% of the time, and it incorrectly detects cancer (in patients who don't have it) only 1% of the time. The test results come back positive. What's the chance that you have cancer?

Bayes' rule says that the answer is precisely a 1 in 102 chance, which is a probability a little below 1%. The remarkable thing about this is that there is only one answer: the odds of you having that type of cancer, given the above information, is exactly 1 in 102; no more, no less.

This is one of the key insights of Bayes' rule: Given what you knew, and what you saw, the maximally accurate state of belief for you to be in is completely pinned down. While that belief state is quite difficult to find in practice, we know how to find it in principle. If you want your beliefs to become more accurate as you observe the world, Bayes' rule gives some hints about what you need to do.

Learn Bayes' rule

• Bayes' rule: Odds form. Bayes' rule is simple, if you think in terms of relative odds.
• Bayes' rule: Proportional form. The fastest way to say something both convincing and true about belief-updating.
• Bayes' rule: Log-odds form. A simple transformation of Bayes' rule reveals tools for measuring degree of belief, and strength of evidence.
• Bayes' rule: Probabilistic form. The original formulation of Bayes' rule.
• Bayes' rule: Functional form. Bayes' rule for continuous variables.
• Bayes' rule: Vector form. For when you want to apply Bayes' rule to lots of evidence and lots of variables, all in one go.

Implications of Bayes' rule

• A Bayesian view of scientific virtues. Why is it that science relies on bold, precise, and falsifiable predictions? Because of Bayes' rule, of course.
• Update by inches. It's virtuous to change your mind in response to overwhelming evidence. It's even more virtuous to shift your beliefs a little bit at a time, in response to all evidence (no matter how small).
• Belief revision as probability elimination. Update your beliefs by throwing away large chunks of probability mass.
• Shift towards the hypothesis of least surprise. When you see new evidence, ask: which hypothesis is least surprised?
• Extraordinary claims require extraordinary evidence. The people who adamantly claim they were abducted by aliens do provide some evidence for aliens. They just don't provide quantitatively enough evidence.
• Ideal reasoning via Bayes' rule. Bayes' rule is to reasoning as the Carnot cycle is to engines: Nobody can be a perfect Bayesian, but Bayesian reasoning is still the theoretical ideal.

Related content

• Subjective probability. Probability is in the mind, not the world. If you don't know whether a tossed coin came up heads or tails, that's a fact about you, not a fact about the coin.
• Probability theory. The quantification and study of objects that represent uncertainty about the world, and methods for making those representations more accurate.
• Information theory. The quantification and study of information, communication, and what it means for one object to tell us about another.

Other articles and introductions

• Wikipedia
• Better Explained