AI ALIGNMENT FORUM
AF

Wikitags

The Utility Function Is Not Up For Grabs

Edited by Zack_M_Davis, Vladimir_Nesov last updated 5th May 2012

The constraints of decision theory are only there to help you win; they don't specify what constitutes a win. Rationality in and of itself cannot constrain what you want, except insofar as what you thought you wanted failed to reflect what you actually wanted (or was just plain inconsistent). Hence the saying: the utility function is not up for grabs.

Blog posts

  • Newcomb's Problem and Regret of Rationality
  • Something to Protect
  • Rationality is Systematized Winning

See also

  • Rationalists should win
  • Something to protect
Discussion0
Discussion0