In short: There are many methods to pool forecasts. The most commonly used is the arithmetic mean of probabilities. However, there are empirical and theoretical reasons to prefer the geometric mean of the odds instead. This is particularly important when some of the predictions have extreme values. Therefore, I recommend defaulting to the geometric mean of odds to aggregate probabilities.

I wrote previously about this topic in LessWrong, and my thoughts have matured quite a bit since then. If you are interested in forecasting, I think you will find this an interesting read. Read more in this link.

New to LessWrong?

New Comment
1 comment, sorted by Click to highlight new comments since: Today at 12:40 AM

When aggregating data, selection of aggregation method always depends upon the answer to the question "for what purpose?"

If you include one extreme outlier prediction, it can radically shift the geometric mean of a bunch of moderate ones. Is this a desirable property for your purposes?

For example: if three people all predict that Ms Green will win something versus Dr Blue with 1:1 odds, and I predict that Dr Blue has one in a million chance, then the arithmetic mean of probabilities says that between us, we think that Dr Blue has about 38% chance. Geometric mean of odds says that we think Dr Blue has 3% chance. Is either of these more useful to you for some purpose than the other?