AI ALIGNMENT FORUM
AF

582
Wikitags

Superrationality

Edited by abramdemski, et al. last updated 17th Nov 2022

Superrationality is a concept invented by Douglas Hofstadter. He thought that agents should cooperate in Prisoner's Dilemma, but the primary notion of "rationality" which had been deeply developed by economists, decision-theorists, and game-theorists disagreed. Rather than fighting over the definition of rational, Douglas Hofstadter coined the term superrational for the kind of rationality he was interested in.

Douglas Hofstadter did not publish a full theory of superrationality, instead focusing on the prisoner's dilemma and other specific examples to communicate his intuition. However, some principles can be clearly inferred from his writing, such as reasoning as if you have control over the other player's policy (in symmetric games).

Eliezer Yudkowsky shared the same core intuition with Douglas Hofstadter, but took the path of trying to reclaim the word rational for what he meant, in Functional Decision Theory.  As a result, LessWrong does not consistently use superrational/superrationality.

Subscribe
Discussion
1
Subscribe
Discussion
1
Posts tagged Superrationality
22Superrational Agents Kelly Bet Influence!
abramdemski
4y
6
21Superrationality in arbitrary games
Vanessa Kosoy
10y
6
13Breaking Oracles: superrationality and acausal trade
Stuart_Armstrong
6y
4
16Infra-Bayesian haggling
hannagabor
1y
0
Add Posts