AI ALIGNMENT FORUM
AF

Nominated Posts for the 2019 Review

Posts need at least 2 nominations to continue into the Review Phase.

Nominate posts that you have personally found useful and important.

Sort by: fewest nominations

20AGI will drastically increase economies of scale

4y

15

1 • 0

20How common is it for one entity to have a 3+ year technological lead on its nearest competitor?

Daniel Kokotajlo

4y

1

1 • 0

49The Commitment Races problem

Daniel Kokotajlo

4y

36

1 • 0

25Misconceptions about continuous takeoff

Matthew Barnett

4y

12

1 • 0

28But exactly how complex and fragile?

4y

30

2 • 1

32The strategy-stealing assumption

Paul Christiano

4y

45

2 • 3

51Gradient hacking

4y

33

2 • 2

29Reframing Impact

4y

4

2 • 1

38Reframing Superintelligence: Comprehensive AI Services as General Intelligence

5y

29

2 • 2

35The Credit Assignment Problem

4y

32

2 • 1

45AI Safety "Success Stories"

4y

11

2 • 1

50Thoughts on Human Models

Ramana Kumar, Scott Garrabrant

5y

9

2 • 1

39Soft takeoff can still lead to decisive strategic advantage

Daniel Kokotajlo

4y

34

2 • 4

53Selection vs Control

4y

14

2 • 2

50Utility ≠ Reward

Vladimir Mikulik

4y

16

2 • 2

25Strategic implications of AIs' ability to coordinate at low cost, for example by merging

5y

22

2 • 1

27Six AI Risk/Strategy Ideas

4y

14

2 • 1

26Classifying specification problems as variants of Goodhart's Law

Victoria Krakovna

4y

5

2 • 1

63Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More

4y

17

2 • 2

56Alignment Research Field Guide

5y

6

2 • 2

96What failure looks like

Paul Christiano

5y

28

2 • 2

55Seeking Power is Often Convergently Instrumental in MDPs

Alex Turner, Logan Riggs Smith

4y

34

2 • 2

47Why Subagents?

4y

12

2 • 1

52Evolution of Modularity

4y

6

2 • 1

66Chris Olah’s views on AGI safety

4y

30

2 • 2

51Understanding “Deep Double Descent”

4y

35

3 • 4

58Risks from Learned Optimization: Introduction

Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, Scott Garrabrant

5y

33

3 • 3

87The Parable of Predict-O-Matic

4y

16

5 • 4

2019 Review Discussion

Soft takeoff can still lead to decisive strategic advantage

39

Daniel Kokotajlo

Industrial RevolutionAI RiskAI TakeoffAI Governance

Frontpage

4y

[Epistemic status: Argument by analogy to historical cases. Best case scenario it's just one argument among many. Edit: Also, thanks to feedback from others, especially Paul, I intend to write a significantly improved version of this post in the next two weeks. Edit: I never did, because in the course of writing my response I realized the original argument made a big mistake. See this review.]

I have on several occasions heard people say things like this:

The original Bostrom/Yudkowsky paradigm envisioned a single AI built by a single AI project, undergoing intelligence explosion all by itself and attaining a decisive strategic advantage as a result. However, this is very unrealistic. Discontinuous jumps in technological capability are very rare, and it is very implausible that one project

...

(Continue Reading – 2200 more words)

Ryan Greenblatt1mo1

It seems like there are strong reasons to expect that the post AI coalitions will look very different from the current world economy, though I agree that they might look like a world economy. For instance, imagine world GDP grows by 100x. It seems totally plausible that Google/TSMC/OpenAI revenue grows by 50x relative to typical other companies which only 2x revenue.

Then, power structures might be dramatically different from current power structures. (Even if the US Government is effectively co-running AI lab(s), I still expect that the power structures ... (read more)