Summary

Bayes's Rule quantifies the notion of 'extraordinary claim' and 'extraordinary evidence'. An extraordinary claim is one with a low prior probability in advance of considering the evidence. Extraordinary evidence is evidence with an extreme likelihood ratio favoring the claim.

The likelihood ratio is defined as:

$\text{Likelihood ratio}=\frac{\text{Probability of seeing the evidence, assuming the claim is true}}{\text{Probability of seeing the evidence, assuming the claim is false}}$

To obtain an extreme likelihood ratio, we mainly need a very low denominator rather than a very high numerator. It does no good to observe that the likelihood of the evidence, given the claim, is very high. What we need is to see that the likelihood of the evidence, given the claim's falsity, is very low.

So to get extremely strong evidence, we need to see an observation which is very improbable, given "business as usual" or plausible alternative hypotheses to the claim. This observation would be deserving of the title, "extraordinary evidence".

A corollary to "extraordinary claims require extraordinary evidence" was stated by Gwern Branwen as "ordinary claims require ordinary evidence". This corollary may be much more important in ordinary life, where we encounter more ordinary claims than extraordinary claims, but may be more tempted to reject them by demanding extraordinary evidence (motivated skepticism).

Example of an extraordinary claim and ordinary evidence: Bookcase aliens.

Consider the following hypothesis: What if there are Bookcase Aliens who teleport into our houses at night and drop off bookcases?

Bob offers the following evidence for this claim: "Last week, I visited my friend's house, and there was a new bookcase there. If there were no bookcase aliens, I wouldn't have expected that my friend would get a new bookcase. But if there are Bookcase Aliens, then the probability of my finding a new bookcase there was much higher. Therefore, my observation, 'There is a new bookcase in my friend's house,' is strong evidence supporting the existence of Bookcase Aliens."

In an intuitive sense, we have a notion that Bob's evidence "There is a new bookcase in my friend's house" is not as extraordinary as the claim "There are bookcase aliens" - that the evidence fails to lift the claim. Bayes's Rule makes this statement precise.

Bob is in fact correct that his observation, "There's a new bookcase in my friend's house", is indeed evidence favoring the Bookcase Aliens. Depending on how long it's been since Bob last visited that house, there might ceteris paribus be, say, a 1% chance that there would be a new bookcase there. On the other hand, the Bookcase Aliens hypothesis might assign, say, 50% probability that the Bookcase Aliens would target this particular house among others. If so, that's a likelihood ratio of 50:1 favoring the Bookcase Aliens hypothesis.

However, a reasonable prior on Bookcase Aliens would assign this a very low prior probability given our other, previous observations of the world. Let's be conservative and assign odds of just 1 : 1,000,000,000 against Bookcase Aliens. Then to raise our posterior belief in Bookcase Aliens to somewhere in the "pragmatically noticeable" range of 1 : 100, we'd need to see evidence with a cumulative likelihood ratio of 10,000,000 : 1 favoring the Bookcase Aliens. 50 : 1 won't cut it.

What would need to change for the observation "There's a new bookcase in my friend's house" to be convincing evidence of Bookcase Aliens, compared to the alternative hypothesis of "business as usual"?

As suggested by the Bayesian interpretation of strength of evidence, what we need to see is an observation which is nigh-impossible if there are not bookcase aliens. We would have to believe that, conditional on "business as usual" being true, the likelihood of seeing a bookcase was on the order of 0.00000001%. That would then take the likelihood ratio, aka strength of evidence, into the rough vicinity of a billion to one favoring Bookcase Aliens over "business as usual".

We would still need to consider whether there might be other alternative hypotheses besides Bookcase Aliens and "business as usual", such as a human-operated Bookcase Conspiracy, but at least we wouldn't be dealing with an observation that was so unsurprising (conditional on business as usual) as to be unable to support any kind of extraordinary claim.

However, if instead we suppose that Bookcase Aliens are allegedly 99.999999% probable to add a bookcase to Bob's friend's house, very little changes - the likelihood ratio is 99.99999% : 1% or 100 : 1 instead of 50 : 1. This is why, to obtain an extreme likelihood ratio, we mainly need a tiny denominator rather than a big numerator. In other words, "extraordinary evidence".

Example of an ordinary claim not requiring extraordinary evidence: A failing new employee.

One contributed example: "A few years back, a senior person at my workplace told me that a new employee wasn't getting his work done on time, and that she'd had to micromanage him to get any work out of him at all. This was an unpleasant fact for a number of reasons; I'd liked the guy, and I'd advocated for hiring him to our Board of Directors just a few weeks earlier (which is why the senior manager was talking to me). I could have demanded more evidence, I could have demanded that we give him more time to work out, I could have demanded a videotape and signed affidavits… but a new employee not working out, just isn't that improbable. Could I have named the exact prior odds of an employee not working out, could I have said how much more likely I was to hear that exact report of a long-term-bad employee than a long-term-good employee? No, but 'somebody hires the wrong person' happens all the time, and I'd seen it happen before. It wasn't an extraordinary claim, and I wasn't licensed to ask for extraordinary evidence. To put numbers on it, I thought the proportion of bad to good employees was on the order of 1 : 4 at the time, and the likelihood ratio for the manager's report seemed more like 10 : 1."

Or to put it another way: The rule is 'extraordinary claims require extraordinary evidence', not 'inconvenient but ordinary claims require extraordinary evidence'.

Extraordinary claims

An obvious next question is just what makes a claim 'extraordinary' or 'ordinary'. This is a deep separate topic, but as an example, consider the claim that the Earth is becoming warmer due to carbon dioxide being added to its atmosphere.

To evaluate the ordinariness or extraordinariness of this claim:

• We don't ask whether the future consequences of this claim seem extreme or important.
• We don't ask whether the policies that would be required to address the claim are very costly.
• We do ask whether "carbon dioxide warms the atmosphere" or "carbon dioxide fails to warm the atmosphere" seems to conform better to the deep, causal generalizations we have about carbon dioxide and heat.
• If we've already considered the deep causal generalizations like those, we don't ask about generalizations causally downstream of the deep causal ones we've already considered, like "Have I ever walked outside and seen the Earth's temperature that high before?"
• We do ask whether learning that carbon dioxide was warming the atmosphere, or alternatively, learning that carbon dioxide was not warming the atmosphere, would imply that we had learned some new fact unknown to modern science. (E.g., while there are possibilities where adding carbon dioxide doesn't warm the Earth, to find out that one of these possibilities were true, would tell us about some previously unknown fact; conversely, we wouldn't learn anything unexpected if the Earth did warm up.)
• We don't consider the lots and lots of carbon dioxide molecules supposedly in the atmosphere to be "additional entities" penalized by Occam's Razor - the fact that this claim involves a large amount of physical stuff doesn't make it extraordinary.
• We don't consider the fact that lots of prestigious scientists believe it. This could be regarded as evidence if you think prestigious scientists are substantially less likely to believe the claim if it is false than if it is true, but it wouldn't factor into the prior probability of the claim.
• We don't care if lots of crazy people clearly believe the claim for bad reasons, or if there's a surface generalization about lots of people believing vaguely similar but false things about other human activities harming the Earth. Reality doesn't have a privileged tendency to avoid facts that are arguably similar to false things people already believe. We might consider this point when asking if we should believe as evidence the observation of someone claiming to have psychically foreseen a hot future, but it doesn't enter into the prior probability that carbon dioxide is warming the atmosphere.

These tests suggest that "Large amounts of added carbon dioxide will incrementally warm Earth's atmosphere" would have been an 'ordinary' claim in advance of trying to find any evidence for or against it - it's just how you would expect a greenhouse gas to work, more or less. Thus, one is not entitled to demand a digitally signed video recordings of carbon dioxide levels in 650 million BCE before believing it.

Start Reading