x

AI ALIGNMENT FORUM
AF

Matthew Barnett — AI Alignment Forum

Matthew Barnett

Top postsTop post

Matthew Barnett

Message

Someone who is interested in learning and doing good.

My Twitter: https://twitter.com/MatthewJBar

My Substack: https://matthewbarnett.substack.com/

10860

Ω

1016

58

1126

2

8y

Matthew Barnett

Someone who is interested in learning and doing good.

My Twitter: https://twitter.com/MatthewJBar

My Substack: https://matthewbarnett.substack.com/

Top postsTop post

Evaluating the historical value misspecification argument

ETA: I'm not saying that MIRI thought AIs wouldn't understand human values. If there's only one thing you take away from this post, please don't take away that. Here is Linch's attempted summary of this post, which I largely agree with. Recently, many people have talked about whether some of the main MIRI people (Eliezer Yudkowsky, Nate Soares, and Rob Bensinger[1]) should update on whether value alignment is easier than they thought given that GPT-4 seems to follow human directions and act within moral constraints pretty well (here are two specific examples of people talking about this: 1, 2). Because these conversations are often hard to follow without much context, I'll just provide a brief caricature of how I think this argument has gone in the places I've seen it, which admittedly could be unfair to MIRI[2]. Then I'll offer my opinion that, overall, I think MIRI people should probably update in the direction of alignment being easier than they thought in light of this information, despite their objections. Note: I encourage you to read this post carefully to understand my thesis. This topic can be confusing, and there are many ways to misread what I'm saying. Also, make sure to read the footnotes if you're skeptical of some of my claims. Here's my very rough caricature of the discussion so far, plus my response: Non-MIRI people: Yudkowsky talked a great deal in the sequences about how it was hard to get an AI to understand human values. For example, his essay on the Hidden Complexity of Wishes made it sound like it would be really hard to get an AI to understand common sense. In that essay, the genie did silly things like throwing your mother out of the building rather than safely carrying her out. Actually, it turned out that it was pretty easy to get an AI to understand common sense. LLMs are essentially safe-ish genies that do what you intend. MIRI people should update on this information. MIRI people (Eliezer Yudkowsky, Nate Soares, and Rob Bensinger):

My thoughts on the social response to AI risk

A proposed method for forecasting transformative AI

122Feb 10, 2023

Misconceptions about continuous takeoff

My thoughts on the social response to AI risk

A common theme implicit in many AI risk stories has been that broader society will either fail to anticipate the risks of AI until it is too late, or do little to address those risks in a serious manner. In my opinion, there are now clear signs that this assumption...

Nov 1, 2023•146

Evaluating the historical value misspecification argument

ETA: I'm not saying that MIRI thought AIs wouldn't understand human values. If there's only one thing you take away from this post, please don't take away that. Here is Linch's attempted summary of this post, which I largely agree with. Recently, many people have talked about whether some of...

Oct 5, 2023•192

Updating Drexler's CAIS model

Eric Drexler's report Reframing Superintelligence: Comprehensive AI Services (CAIS) as General Intelligence reshaped how a lot of people think about AI (summary 1, summary 2). I still agree with many parts of it, perhaps even the core elements of the model. However, after looking back on it more than four...

Jun 16, 2023•47

A proposed method for forecasting transformative AI

In 2021, I proposed measuring progress in the perplexity of language models and extrapolating past results to determine when language models were expected to reach roughly "human-level" performance. Here, I build on that approach by introducing a more systematic and precise method of forecasting progress in language modeling that employs...

Feb 10, 2023•122

A conversation about Katja's counterarguments to AI risk

This post is a transcript of a conversation between Ege Erdil and Ronny Fernandez, recorded by me. The participants talked about a recent post by Katja Grace that presented many counterarguments to the basic case for AI x-risk. You might want to read that post first. As it was transcribed...

Oct 18, 2022•43

A comment on Ajeya Cotra's draft report on AI timelines

Ajeya Cotra's draft report on AI timelines is the most useful, comprehensive report about AI timelines I've seen so far. I think the report is a big step towards modeling the fundamental determinants of AI progress. That said, I am skeptical that the arguments provided in the report should reduce...

Feb 24, 2022•74

Forecasting progress in language models

Note: this post was cross-posted to Metaculus over a week ago as part of their new Metaculus journal. Here, I describe a way of measuring the performance of language models, and extrapolate this measure using publicly available data on benchmarks. The result is a (surprisingly) short timeline to "human-level"—within one...

Oct 28, 2021•62

Load More (7/19)