x

AI ALIGNMENT FORUM
AF

Charlie Steiner — AI Alignment Forum

Charlie Steiner

Top postsTop post

Charlie Steiner

Message

If you want to chat, message me!

LW1.0 username Manfred. PhD in condensed matter physics. I am independently thinking and writing about value learning.

8504

Ω

1172

75

2390

8y

Charlie Steiner

If you want to chat, message me!

LW1.0 username Manfred. PhD in condensed matter physics. I am independently thinking and writing about value learning.

Top postsTop post

Some background for reasoning about dual-use alignment research

This is pretty basic. But I still made a bunch of mistakes when writing this, so maybe it's worth writing. This is background to a specific case I'll put in the next post. It's like a a tech tree If we're looking at the big picture, then whether some piece of research is net positive or net negative isn't an inherent property of that research; it depends on how that research is situated in the research ecosystem that will eventually develop superintelligent AI. A tech tree, with progress going left to right. Blue research is academic, green makes you money, red is a bad ending, yellow is a good ending. Stronger connections are more important prerequisites. Consider this toy game in the picture. We start at the left and can unlock technologies, with unlocks going faster the stronger our connections to prerequisites. The red and yellow technologies in the picture are superintelligent AI - pretend that as soon as one of those technologies is unlocked, the hastiest fraction of AI researchers are immediately going to start building it. Your goal is for humanity to unlock yellow technology before a red one. This game would be trivial if everyone agreed with you. But there are many people doing research, and they have all kinds of motivations - some want as many nodes to be unlocked as possible (pure research - blue), some want to personally unlock a green node (profit - green), some want to unlock the nearest red or yellow node no matter which it is (blind haste - red), and some want the same thing as you (beneficial AI - yellow) but you have a hard time coordinating with them. In this baseline tech tree game, it's pretty easy to play well. If you're strong, just take the shortest path to a yellow node that doesn't pass too close to any red nodes. If you're weak, identify where the dominant paradigm is likely to end up, and do research that differentially advantages yellow nodes in that future. The tech tree is wrinkly But of course there are lots of wrinkles not i

127May 18, 2023

Neural uncertainty estimation review article (for alignment)

Take 13: RLHF bad, conditioning good.

[Simulators seminar sequence] #1 Background & shared assumptions

Neural uncertainty estimation review article (for alignment)

EDIT 1/27: This post neglects the entire sub-field of estimating uncertainty of learned representations, as in https://openreview.net/pdf?id=e9n4JjkmXZ. I might give that a separate follow-up post. Introduction Suppose you've built some AI model of human values. You input a situation, and it spits out a goodness rating. You might want to...

Dec 5, 2023•74

How to solve deception and still fail.

A mostly finished post I'm kicking out the door. You'll get the gist. I There's a tempting picture of alignment that centers on the feeling of "As long as humans stay in control, it will be okay." Humans staying in control, in this picture, is something like humans giving lots...

Oct 4, 2023•43

Some background for reasoning about dual-use alignment research

This is pretty basic. But I still made a bunch of mistakes when writing this, so maybe it's worth writing. This is background to a specific case I'll put in the next post. It's like a a tech tree If we're looking at the big picture, then whether some piece...

May 18, 2023•127

[Simulators seminar sequence] #2 Semiotic physics - revamped

Update February 21st: After the initial publication of this article (January 3rd) we received a lot of feedback and several people pointed out that propositions 1 and 2 were incorrect as stated. That was unfortunate as it distracted from the broader arguments in the article and I (Jan K) take...

Feb 27, 2023•24

Shard theory alignment has important, often-overlooked free parameters.

A delayed hot take. This is pretty similar to previous comments from Rohin. Shard theory alignment requires magic - not in the sense of magic spells, but in the technical sense of steps we need to remind ourselves we don't know how to do. Locating magic is an important step...

Jan 20, 2023•37

[Simulators seminar sequence] #1 Background & shared assumptions

Meta: Over the past few months, we've held a seminar series on the Simulators theory by janus. As the theory is actively under development, the purpose of the series is to discover central structures and open problems. Our aim with this sequence is to share some of our discussions with...

Jan 2, 2023•50

Take 14: Corrigibility isn't that great.

As a writing exercise, I'm writing an AI Alignment Hot Take Advent Calendar - one new hot take, written every day some days for 25 days. It's the end (I saved a tenuous one for ya')! Kind of disappointing that this ended up averaging out to one every 2 days,...

Dec 25, 2022•15

Load More (7/46)