AI ALIGNMENT FORUM
AF

Wikitags

Quantilization

Edited by plex, Mateusz Bagiński, et al. last updated 2nd Dec 2024

A Quantilizer is a proposed AI design that aims to reduce the harms from Goodhart's law and specification gaming by selecting reasonably effective actions from a distribution of human-like actions, rather than maximizing over actions. It is more of a theoretical tool for exploring ways around these problems than a practical buildable design.

See also

  • Quantilizers: AI That Doesn't Try Too Hard by Rob Miles
  • Arbital page on Quantilizers
  • Quantilizers: A Safer Alternative to Maximizers for Limited Optimization by Jessica Taylor (original paper)
Subscribe
1
Subscribe
1
Discussion0
Discussion0
Posts tagged Quantilization
13Quantilizers maximize expected utility subject to a conservative cost constraint
jessicata
10y
3
10Another view of quantilizers: avoiding Goodhart's Law
jessicata
10y
1
27When to use quantilization
RyanCarey
7y
0
10Quantilal control for finite MDPs
Vanessa Kosoy
7y
0
6Computing an exact quantilal policy
Vanessa Kosoy
7y
0
43Soft optimization makes the value target bigger
Jeremy Gillen
3y
4
14Quantilizers and Generative Models
Adam Jermyn
3y
5
8Why don't quantilizers also cut off the upper end of the distribution?
Q
Alex_Altair, Jeremy Gillen
2y
Q
1
19[Aspiration-based designs] 1. Informal introduction
B Jacobs, Jobst Heitzig, Simon Fischer, Simon Dima
1y
0
16The murderous shortcut: a toy model of instrumental convergence
Thomas Kwa
1y
0
10Hedonic Loops and Taming RL
beren
2y
0
5Quantilizer ≡ Optimizer with a Bounded Amount of Output
itaibn0
4y
0
15How to safely use an optimizer
Simon Fischer
1y
0
20Recursive Quantilizers II
abramdemski
5y
15
9Stable Pointers to Value III: Recursive Quantilization
abramdemski
7y
0
Load More (15/16)
Add Posts