Edouard Harris

AI takeoff story: a continuation of progress by other means

Thanks to Vladimir Mikulik for suggesting that I write this, and to Rohin Shah and Daniel Kokotajlo for kindly providing feedback. Prologue This is a story about a universe a lot like ours. In this universe, the scaling hypothesis — which very roughly says that you can make an AI smarter just by making it bigger — turns out to be completely right. It’s gradually realized that advances in AI don’t arise from conceptual breakthroughs or sophisticated deep learning architectures. Just the opposite: the simpler the architecture, the better it turns out to perform at scale. Past a certain point, clever model-building was just slowing down progress. Researchers in this universe discover a rough rule of thumb: each neural network architecture has an intrinsic maximum potential intelligence, or “capability”. When you train a network on a problem, how close it gets to reaching its potential capability depends on two limiting factors: 1) the size and diversity of its dataset; and 2) the amount of compute that’s used to train it. Training a network on a quadrillion games of tic-tac-toe won’t make it smart, but training a network on a quadrillion-word corpus of text might just do it. Even data cleaning and quality control don’t matter too much: as long as you have scale, if you train your system long enough, it learns to separate signal from noise automatically. Generally, the more parameters a neural network has, the higher its potential capability. Neural nets with simple architectures also have a higher potential capability than nets with more sophisticated architectures do. This last observation takes the research community longer to absorb than you might expect — it’s a bitter lesson, after all — so the groups that internalize it first have an early edge. Frontier AI projects begin to deemphasize architecture innovations and any but the most basic data preprocessing. They focus instead on simple models, huge datasets, hard problems, and abundant compute. Initial prog

76Sep 27, 2021

Edouard Harris

Message

Co-founder @ Gladstone AI.

Contact: edouard@gladstone.ai

Website: eharr.is

593

251

Inverse scaling can become U-shaped

Edit: Here's a great comment by Ethan Perez that caveats this result, that I'd recommend reading for context. This is a paper by folks at Quoc Le's team at Google that examines the winning tasks from Round 1 of Anthropic's Inverse Scaling Prize. They find that 3/4 of the winning...

Nov 8, 202227

POWERplay: An open-source toolchain to study AI power-seeking

We're open-sourcing POWERplay, a research toolchain you can use to study power-seeking behavior in reinforcement learning agents. POWERplay was developed by Gladstone AI for internal research. POWERplay's main use is to estimate the instrumental value that a reinforcement learning agent can get from a state in an MDP. Its implementation...

Oct 24, 202229

Instrumental convergence: scale and physical interactions

Summary of this post This is the third post in a three-part sequence on instrumental convergence in multi-agent RL. Read Part 1 and Part 2. In this post, we’ll: 1. Investigate instrumental convergence on a multi-agent gridworld with a complicated topology. 2. Show that when we add a simple physical...

Oct 14, 202222

Misalignment-by-default in multi-agent systems

Summary of this post This is the second post in a three-part sequence on instrumental convergence in multi-agent RL. Read Part 1 here. In this post, we’ll: 1. Define formal multi-agent POWER (i.e., instrumental value) in a setting that contains a "human" agent and an "AI" agent. 2. Introduce the...

Oct 13, 202221

Instrumental convergence in single-agent systems

Summary of the sequence Over the past few months, we’ve been investigating instrumental convergence in reinforcement learning agents. We started from the definition of single-agent POWER proposed by Alex Turner et al., extended it to a family of multi-agent scenarios that seemed relevant to AI alignment, and explored its implications...

Oct 12, 202233

AI Tracker: monitoring current and near-future risks from superscale models

TLDR: We've put together a website to track recent releases of superscale models, and comment on the immediate and near-term safety risks they may pose. The website is little more than a view of an Airtable spreadsheet at the moment, but we'd greatly appreciate any feedback you might have on...

Nov 23, 202167

AI takeoff story: a continuation of progress by other means

Sep 27, 202176

Load More (7/8)

AI ALIGNMENT FORUM
AF

AI ALIGNMENT FORUM
AF

Edouard Harris

Edouard Harris

Edouard Harris

AI takeoff story: a continuation of progress by other means

AI Tracker: monitoring current and near-future risks from superscale models

Instrumental convergence in single-agent systems

POWERplay: An open-source toolchain to study AI power-seeking

Edouard Harris

Inverse scaling can become U-shaped

POWERplay: An open-source toolchain to study AI power-seeking

Instrumental convergence: scale and physical interactions

Misalignment-by-default in multi-agent systems

Instrumental convergence in single-agent systems

AI Tracker: monitoring current and near-future risks from superscale models

AI takeoff story: a continuation of progress by other means

AI takeoff story: a continuation of progress by other means

AI Tracker: monitoring current and near-future risks from superscale models

Instrumental convergence in single-agent systems

POWERplay: An open-source toolchain to study AI power-seeking

Inverse scaling can become U-shaped

POWERplay: An open-source toolchain to study AI power-seeking

Instrumental convergence: scale and physical interactions

Misalignment-by-default in multi-agent systems

Instrumental convergence in single-agent systems

AI Tracker: monitoring current and near-future risks from superscale models

AI takeoff story: a continuation of progress by other means