The Hanson-Yudkowsky AI-Foom Debate

In late 2008, an extensive and long-awaited debate about the Technological Singularity occurred on , mainly between and . It focused on the likelihood of hard AI takeoff ("FOOM"), the need for a theory of Friendliness, and the future of AGI, , and in general. A stand-up debate on the same themes took place at Jane Street Capital in June 2011, and Hanson made a further post in 2014.

See Also

The best resource for most readers new to the debate will be MIRI's 2013 The Hanson-Yudkowsky AI-Foom Debate eBook, available in PDF, EPUB, and MOBI formats. This includes

• the original series of blog posts,
• a transcript of the 2011 debate
• Kaj Sotala's summary and analysis of the debate, and
• a 2013 technical report on AI takeoff dynamics (“intelligence explosion microeconomics”) written by Yudkowsky.

Prologue

• Fund UberTool? by
• Engelbart as UberTool? by
• Friendly Teams by
• Friendliness Factors by
• The Weak Inside View by
• Setting The Stage by
• The First World Takeover by
• Abstraction, Not Analogy by
• Whence Your Abstractions? by

Main sequence

• AI Go Foom by
• Optimization and the Singularity by
• Eliezer's Meta-Level Determinism by
• Observing Optimization by
• Life's Story Continues by
• Emulations Go Foom by
• Brain Emulation and Hard Takeoff by
• Billion Dollar Bots by James Miller
• Surprised by Brains by
• “Evicting” brain emulations by
• Cascades, Cycles, Insight... by
• When Life Is Cheap, Death Is Cheap by
• ...Recursion, Magic by
• Abstract/Distant Future Bias by
• Engelbart: Insufficiently Recursive by
• Total Nano Domination by
• Dreams of Autarky by
• Total Tech Wars by
• Singletons Rule OK by
• Stuck In Throat by
• Disappointment in the Future by
• I Heart CYC by
• Is the City-ularity Near? by
• Recursive Self-Improvement by
• Whither Manufacturing? by
• Hard Takeoff by
• Test Near, Apply Far by
• Permitted Possibilities, & Locality by
• Underconstrained Abstractions by
• Beware Hockey Stick Plans by
• Evolved Desires by
• Sustained Strong Recursion by
• Friendly Projects vs. Products by
• Is That Your True Rejection? by
• Shared AI Wins by
• Artificial Mysterious Intelligence by
• Wrapping Up by
• True Sources of Disagreement by
• The Bad Guy Bias by
• Disjunctions, Antipredictions, Etc. by
• Are AIs Homo Economicus? by
• Two Visions Of Heritage by
• The Mechanics of Disagreement by

Conclusion

• What Core Argument? by
• What I Think, If Not Why by
• Not Taking Over the World by

Postscript

• We Agree: Get Froze by
• You Only Live Twice by
• Debating Yudkowsky by
• Foom Debate, Again by

Jane Street debate

A stand-up debate between Hanson and Yudkowsky on the same themes took place on Wednesday June 30 2011 at Jane Street Capital.

• Video with better quality audio 1.57G
• Poor quality WMA audio 97M (dead link)
• Poor quality MP3 audio 97M (dead link)
• Full transcript
• Partial transcript by Chris Hallquist

Summary and analysis

In Spring 2012 asked Kaj Sotala to write a summary and analysis of the debate; he made a draft of that analysis available in November 2012. The final version of the summary is included in the ebook.

Post-book material

Not included in MIRI's 2013 book on the debate:

• Robin Hanson's 2014 blog post on the subject in response to the publication of "Superintelligence", I Still Don’t Get Foom
• Katja Grace's 2014 hosted discussion on Less Wrong, Superintelligence 6: Intelligence explosion kinetics
• Eliezer's 2017 post, AlphaGo Zero and the Foom Debate and Robin Hanson's reply, What Evidence Is AlphaGo Zero Re AGI Complexity?