- Bickel, E.J. (2007). "Some Comparisons among Quadratic, Spherical, and Logarithmic Scoring Rules". Decision Analysis, 4, (2), 49–65.
Tilmann Gneiting; Adrian E Raftery (March 2007). "Strictly Proper Scoring Rules, Prediction, and Estimation". Journal of the American Statistical Association 102 (477): 359-378. (PDF)

- A Technical Explanation of Technical Explanation

•

Created by MrHen at

The logarithmic scoring rule assigns a negative payoff for all outcomes. The higher the score, the better calibrated the system is.

Score = log(abs(outcome - prediction))

where "outcome" is 1 or 0, and "prediction" is the probability on (0, 1) that the system assigned to the outcome that actually occurred.

As an example of a probabilistic prediction, consider a sports magazine dealing with horse races that gives the winning chance of each horse for each race the day before. If we gather data regarding those predictions and compare it to the actual results, we have a measure – a scoring rule - of the magazine’s performance. This scoring is almost always ~~not linear, however,~~nonlinear, and there are many different transformations which are widely used.

The Brier score, for example, can be seen as a cost function. Essentially, it measures the mean squared difference between a set of predictions and the set of actual outcomes. Therefore, the lowest the score, the better calibrated the prediction system is. Its a scoring rule appropriated for binary of multiple discrete categories, but it should be used with ordinal variables. Mathematically, its an affine transformation of the simpler Quadratic scoring rule.

In decision theory, a **scoring rule** is a measure of ~~someone'~~performance of probabilistic predictions - made under uncertainty.

As an example of a probabilistic prediction, consider a sports magazine dealing with horse races that gives the winning chance of each horse for each race the day before. If we gather data regarding those predictions and compare it to the actual results, we have a measure – a scoring rule - of the magazine’s ~~performance at making predictions under uncertainty (which comprises two distinct aspects, calibration~~performance. This scoring is almost always not linear, however, and ~~discrimination). ~~there are many different transformations which are widely used.

A **proper scoring rule** is one that encourages the forecaster to be ~~honest, as~~honest – that is, the expected payoff is maximized by accurately reporting personal ~~belief~~beliefs about the predicted event, rather than by gaming the system. These rules include the Logarithmic scoring rule, Spherical scoring rule and Brier/Quadratic scoring rule.

- Bickel, E.J. (2007). "Some Comparisons among Quadratic, Spherical, and Logarithmic Scoring Rules". Decision Analysis, 4, (2), 49–65.
(PDF)

- A Technical Explanation of Technical Explanation

In decision theory, a **scoring rule** is a measure of someone's performance at making predictions under ~~uncertainty.~~uncertainty (which comprises two distinct aspects, calibration and discrimination). A **proper scoring rule** encourages the forecaster to be honest, as expected payoff is maximized by accurately reporting personal belief about the predicted event, rather than by gaming the system.

In decision theory, a **scoring rule** is a measure of someone's performance at making predictions under uncertainty. A **proper scoring rule** encourages the forecaster to be honest, as expected payoff is maximized by accurately reporting personal belief about the predicted event, rather than by gaming the system.

In decision theory, a **scoring rule** is a measure of someone's performance at making predictions~~ or choosing actions~~ under uncertainty.

~~A Scoring~~In decision theory, a **scoring rule** is a ~~method for determining the accuracies~~measure of someone's performance at making predictions ~~made~~or choosing actions under~~ conditions of~~ uncertainty.

A Scoring rule is a method for determining the accuracies of predictions made under conditions of uncertainty.

The Brier score, for example, can be seen as a cost function. Essentially, it measures the mean squared difference between a set of predictions and the set of actual outcomes. Therefore, the

~~lowest~~lower the score, the better calibrated the prediction system is.~~Its~~It is a scoring rule appropriated for binary of multiple discrete categories, but it should be used with ordinal variables. Mathematically,~~its~~it is an affine transformation of the simpler Quadratic scoring rule.