Jacob Steinhardt

What will GPT-2030 look like?

GPT-4 surprised many people with its abilities at coding, creative brainstorming, letter-writing, and other skills. Surprises in machine learning are not restricted to GPT-4: I was previously surprised by Minerva’s mathematical abilities, as were many competitive forecasters. How can we be less surprised by developments in machine learning? Our brains often implicitly make a zeroth-order forecast: looking at the current state of the art, and adding on improvements that “feel reasonable”. But what “seems reasonable” is prone to cognitive bias, and will underestimate progress in a fast-moving field like ML. A more effective approach is first-order forecasting: quantifying the historical rate of progress and extrapolating it forward, while also considering reasons for possible slowdowns or speedups.[1] In this post, I’ll use this approach to forecast the properties of large pretrained ML systems in 2030. I’ll refer throughout to “GPT2030”, a hypothetical system that has the capabilities, computational resources, and inference speed that we’d project for large language models in 2030 (but which was likely trained on other modalities as well, such as images). To forecast GPT2030’s properties, I consulted a variety of sources, including empirical scaling laws, projections of future compute and data availability, velocity of improvement on specific benchmarks, empirical inference speed of current systems, and possible future improvements in parallelism. GPT2030’s capabilities turn out to be surprising (to me at least). In particular, GPT2030 will enjoy a number of significant advantages over current systems[2], as well as (in at least some important respects) current human workers: 1. GPT2030 will likely be superhuman at various specific tasks, including coding, hacking, and math, and potentially protein engineering (Section 1). 2. GPT2030 can “work” and “think” quickly: I estimate it will be 5x as fast as humans as measured by words processed per minute [range: 0.5x

185Jun 7, 2023

Jacob Steinhardt

Message

5986

487

835

16y

Scalable End-to-End Interpretability

This is partly a linkpost for Predictive Concept Decoders, and partly a response to Neel Nanda's Pragmatic Vision for AI Interpretability and Leo Gao's Ambitious Vision for Interpretability. There is currently somewhat of a debate in the interpretability community between pragmatic interpretability---grounding problems in empirically measurable safety tasks---and ambitious interpretability----obtaining...

Dec 18, 2025117

Approaching Human-Level Forecasting with Language Models

TL;DR: We present a retrieval-augmented LM system that nears the human crowd performance on judgemental forecasting. Paper: https://arxiv.org/abs/2402.18563 (Danny Halawi*, Fred Zhang*, Chen Yueh-Han*, and Jacob Steinhardt) Twitter thread: https://twitter.com/JacobSteinhardt/status/1763243868353622089 Abstract Forecasting future events is important for policy and decision-making. In this work, we study whether language models (LMs) can...

Feb 29, 202460

What will GPT-2030 look like?

Jun 7, 2023185

Emergent Deception and Emergent Optimization

[Note: this post was drafted before Sydney (the Bing chatbot) was released, but Sydney demonstrates some particularly good examples of some of the issues I discuss below. I've therefore added a few Sydney-related notes in relevant places.] I’ve previously argued that machine learning systems often exhibit emergent capabilities, and that...

Feb 20, 202364

Some Lessons Learned from Studying Indirect Object Identification in GPT-2 small

To learn more about this work, check out the paper. We assume general familiarity with transformer circuits. Intro: There isn’t much interpretability work that explains end-to-end how a model is able to do some task (except for toy models). In this work, we make progress towards this goal by understanding...

Oct 28, 2022101

Forecasting ML Benchmarks in 2023

Thanks to Collin Burns, Ruiqi Zhong, Cassidy Laidlaw, Jean-Stanislas Denain, and Erik Jones, who generated most of the considerations discussed in this post. Previously, I evaluated the accuracy of forecasts about performance on the MATH and MMLU (Massive Multitask) datasets. I argued that most people, including myself, significantly underestimated the...

Jul 18, 202236

Early 2022 Paper Round-up

My students and collaborators have been doing some particularly awesome work over the past several months, and to highlight that I wanted to summarize their papers here, and explain why I’m excited about them. There’s six papers in three categories. Human-Aligned AI * The Effects of Reward Misspecification: Mapping and...

Apr 14, 202280

Load More (7/16)

AI ALIGNMENT FORUM
AF

AI ALIGNMENT FORUM
AF

Jacob Steinhardt

Jacob Steinhardt

Jacob Steinhardt

What will GPT-2030 look like?

More Is Different for AI

Future ML Systems Will Be Qualitatively Different

Scalable End-to-End Interpretability

Jacob Steinhardt

Scalable End-to-End Interpretability

Approaching Human-Level Forecasting with Language Models

What will GPT-2030 look like?

Emergent Deception and Emergent Optimization

Some Lessons Learned from Studying Indirect Object Identification in GPT-2 small

Forecasting ML Benchmarks in 2023

Early 2022 Paper Round-up

What will GPT-2030 look like?

More Is Different for AI

Future ML Systems Will Be Qualitatively Different

Scalable End-to-End Interpretability

Scalable End-to-End Interpretability

Approaching Human-Level Forecasting with Language Models

What will GPT-2030 look like?

Emergent Deception and Emergent Optimization

Some Lessons Learned from Studying Indirect Object Identification in GPT-2 small

Forecasting ML Benchmarks in 2023

Early 2022 Paper Round-up