Probability theory is the mathematics governing quantitative degrees of belief.

See Also: Priors, Bayes' Theorem, Mind Projection Fallacy

Bayesian vs Frequentist Interpretations of Probability

Although most of the basics and axioms of probability theory are uncontroversial, the interpretations, usages, and relative importance given to each result vary. There are two main interpretations of the concept of probability: the Bayesian (subjectivist, epistemic or evidential) and the frequentist (objectivist) ¹. The latter was the major and standard view from late 19th century until the late 20th century, when the Bayesian interpretation gained popularity in many fields of science and philosophy².

In the Bayesian interpretation, probability is seem as a belief about the credence of an event³ ⁴, whereas frequentist interpretations hold that probability is an objective property of a physical system, a propensity on some accounts⁵. An event with Bayesian probability of .6 (or 60%) should be interpreted as stating "With confidence 60%, this event contains the true outcome", whereas a frequentist interpretation would view it as stating "Over 100 trials, we should observe event X approximately 60 times." Frequentists tend to base their view of probability in the Law of Large Numbers, holding the expected probability of an individual event to be close to the average of the results obtained from a large number of trials. Thus, only repeatable events can be said to have probabilities.

The Bayesian interpretation, on the other hand, allows one to assign probabilities to both beliefs and events that have never before happened. Bayes' theorem is used to update the probability of those beliefs when presented with new evidence. The Bayesian interpretation aims to model the correct way of thinking rationally when one is forced to deal with ignorance and uncertainty⁶. Many other fields, such as the study of cognitive biases, use it as the golden reference for rationality.

Many philosophers have argued for a complementary view of probability⁷ ⁸ ⁹ ¹⁰, where both interpretations have their places and value. In the paper "A Subjectivist's Guide to Objective Chance", David Lewis has constructed a view where one can incorporate the frequentist interpretation (objective, chance) inside the Bayesian (subjective, credence) as special case. The frequentist probability is a case of a Bayesian probability conditionalized on truth or empirical evidence: "Chance is objectified subjective probability(...). Objectified credence is credence conditional on the truth." ¹¹

Notable Posts

What is Bayesianism?
Probability is Subjectively Objective
Probability is in the Mind
When (Not) To Use Probabilities
An objective defense of Bayesianism
Frequentist Statistics are Frequently Subjective

Notes and References

It’s worth mentioning that within the Bayesian interpretation there are also viewpoints referred to as "objectivist" and "subjectivist". See Probability is Subjectively Objective for more on this.↩
WOLPERT, R.L. (2004) “A conversation with James O. Berger”, Statistical science, 9, 205–218.↩
BERNARDO, J. M. & SMITH, A. F. M. (1994) “Bayesian Theory”. Wiley.↩
JAYNES, E. T. (1996) ”Probability theory: The logic of science.'' Available from http://bayes.wustl.edu/etj/prob.html ↩
POPPER, Karl.(1959) "The propensity interpretation of probability" The British Journal of the Philosophy of Science, Vol. 10, No. 37. (May, 1959), pp.25-42. Available at: http://www.hum.utah.edu/~mhaber/Documents/Course%20Readings/Popper_Propensity_BJPS1959LITE.pdf ↩
KORB, Kevin & NICHOLSON, Ann. (2010) "Bayesian Artificial Intelligence". CRC Press. p. 28↩
CARNAP, Rudolf. (1945). "The Two Concepts of Probability", Philosophy and Phenomenological Research 5, 513-532.↩
JEFFREY, Richard C. (1965). "The Logic of Decision." New York: McGraw-Hill.↩
LEWIS, David. (1980). “A Subjectivist's Guide to Objective Chance”. In JEFFREY, Richard C. (ed.), Studies in Inductive Logic and Probability. University of California Press. Available at: http://fitelson.org/probability/Lewis_asgtoc.pdf ↩
COX, R. T. (1946) “Probability, frequency and reasonable expectation,'' American Journal of Physics, vol. 14, no. 1, pp. 1-13↩