Normalization (probability)

"Normalization" is an arithmetical procedure carried out to obtain a set of probabilities summing to exactly 1, in cases where we believe that exactly one of the corresponding possibilities is true, and we already know the relative probabilities.

For example, suppose that the odds of Alexander Hamilton winning a presidential election are 3 : 2. But Alexander Hamilton must either win or not win, so the probabilities of him winning or not winning should sum to 1. If we just add 3 and 2, however, we get 5, which is an unreasonably large probability.

If we rewrite the odds as 0.6 : 0.4, we've preserved the same proportions, but made the terms sum to 1. We therefore calculate that Hamilton has a 60% probability of winning the election.

We normalized those odds by dividing each of the terms by the sum of terms, i.e., went from 3 : 2 to $\frac{3}{3+2}:\frac{2}{3+2}=0.6:\mathrm{0.4.}$

In converting the odds $m:n$ to $\frac{m}{m+n}:\frac{n}{m+n},$ the factor $\frac{1}{m+n}$ by which we multiply all elements of the ratio is called a normalizing constant.

More generally, if we have a relative-odds function $O\left(H\right)$ where $H$ has many components, and we want to convert this to a probability function $P\left(H\right)$ that sums to 1, we divide every element of $O\left(H\right)$ by the sum of all elements in $O\left(H\right).$ That is:

$P\left(H_i\right)=\frac{O\left(H_i\right)}{{\sum }_{i}O\left(H_i\right)}$

Analogously, if $O\left(x\right)$ is a continuous distribution on $X$, we would normalize it (create a proportional probability function $P\left(x\right)$ whose integral is equal to 1) by dividing $O\left(x\right)$ by its own integral:

$P\left(x\right)=\frac{O\left(x\right)}{\int O\left(x\right)dx}$

In general, whenever a probability function on a variable is proportional to some other function, we can obtain the probability function by normalizing that function:

$P\left(H\right)\propto O\left(H\right)⟹P\left(H\right)=\frac{O\left(H\right)}{\sum O\left(H\right)}$