In late 2008, an extensive and long-awaited debate about the Technological Singularity occurred on Overcoming Bias, mainly between Robin Hanson and Eliezer Yudkowsky. It focused on the likelihood of hard AI takeoff ("FOOM"), the need for a theory of Friendliness, and the future of AGI, whole brain emulation, and recursive self-improvement in general. A stand-up debate on the same themes took place at Jane Street Capital in June 2011, and Hanson made a further post in 2014.

See Also

The best resource for most readers new to the debate will be MIRI's 2013 The Hanson-Yudkowsky AI-Foom Debate eBook, available in PDF, EPUB, and MOBI formats. This includes

the original series of blog posts,
a transcript of the 2011 debate
Kaj Sotala's summary and analysis of the debate, and
a 2013 technical report on AI takeoff dynamics (“intelligence explosion microeconomics”) written by Yudkowsky.

Prologue

Fund UberTool? by Robin Hanson
Engelbart as UberTool? by Robin Hanson
Friendly Teams by Robin Hanson
Friendliness Factors by Robin Hanson
The Weak Inside View by Eliezer Yudkowsky
Setting The Stage by Robin Hanson
The First World Takeover by Eliezer Yudkowsky
Abstraction, Not Analogy by Robin Hanson
Whence Your Abstractions? by Eliezer Yudkowsky

Main sequence

AI Go Foom by Robin Hanson
Optimization and the Singularity by Eliezer Yudkowsky
Eliezer's Meta-Level Determinism by Robin Hanson
Observing Optimization by Eliezer Yudkowsky
Life's Story Continues by Eliezer Yudkowsky
Emulations Go Foom by Robin Hanson
Brain Emulation and Hard Takeoff by Carl Shulman
Billion Dollar Bots by James Miller
Surprised by Brains by Eliezer Yudkowsky
“Evicting” brain emulations by Carl Shulman
Cascades, Cycles, Insight... by Eliezer Yudkowsky
When Life Is Cheap, Death Is Cheap by Robin Hanson
...Recursion, Magic by Eliezer Yudkowsky
Abstract/Distant Future Bias by Robin Hanson
Engelbart: Insufficiently Recursive by Eliezer Yudkowsky
Total Nano Domination by Eliezer Yudkowsky
Dreams of Autarky by Robin Hanson
Total Tech Wars by Robin Hanson
Singletons Rule OK by Eliezer Yudkowsky
Stuck In Throat by Robin Hanson
Disappointment in the Future by Eliezer Yudkowsky
I Heart CYC by Robin Hanson
Is the City-ularity Near? by Robin Hanson
Recursive Self-Improvement by Eliezer Yudkowsky
Whither Manufacturing? by Robin Hanson
Hard Takeoff by Eliezer Yudkowsky
Test Near, Apply Far by Robin Hanson
Permitted Possibilities, & Locality by Eliezer Yudkowsky
Underconstrained Abstractions by Eliezer Yudkowsky
Beware Hockey Stick Plans by Robin Hanson
Evolved Desires by Robin Hanson
Sustained Strong Recursion by Eliezer Yudkowsky
Friendly Projects vs. Products by Robin Hanson
Is That Your True Rejection? by Eliezer Yudkowsky
Shared AI Wins by Robin Hanson
Artificial Mysterious Intelligence by Eliezer Yudkowsky
Wrapping Up by Robin Hanson
True Sources of Disagreement by Eliezer Yudkowsky
The Bad Guy Bias by Robin Hanson
Disjunctions, Antipredictions, Etc. by Eliezer Yudkowsky
Are AIs Homo Economicus? by Robin Hanson
Two Visions Of Heritage by Robin Hanson
The Mechanics of Disagreement by Eliezer Yudkowsky

Conclusion

Postscript

Jane Street debate

A stand-up debate between Hanson and Yudkowsky on the same themes took place on Wednesday June 30 2011 at Jane Street Capital.

Video with better quality audio 1.57G
~~Poor quality WMA audio~~ 97M (dead link)
~~Poor quality MP3 audio~~ 97M (dead link)
Full transcript
Partial transcript by Chris Hallquist

Summary and analysis

In Spring 2012 Luke Muehlhauser asked Kaj Sotala to write a summary and analysis of the debate; he made a draft of that analysis available in November 2012. The final version of the summary is included in the ebook.

Post-book material

Not included in MIRI's 2013 book on the debate:

Robin Hanson's 2014 blog post on the subject in response to the publication of "Superintelligence", I Still Don’t Get Foom
Katja Grace's 2014 hosted discussion on Less Wrong, Superintelligence 6: Intelligence explosion kinetics
Eliezer's 2017 post, AlphaGo Zero and the Foom Debate and Robin Hanson's reply, What Evidence Is AlphaGo Zero Re AGI Complexity?

jonathanberger

jonathanberger