x

AI ALIGNMENT FORUM

AF

Max H

Top postsTop post

Max H

Message

Most of my posts and comments are about AI and alignment. Posts I'm most proud of, which also provide a good introduction to my worldview:

Without a trajectory change, the development of AGI is likely to go badly
Steering systems, and a follow up on corrigibility.
"Aligned" foundation

...

3287

Ω

73

24

433

12y

Max H

Most of my posts and comments are about AI and alignment. Posts I'm most proud of, which also provide a good introduction to my worldview:

Without a trajectory change, the development of AGI is likely to go badly
Steering systems, and a follow up on corrigibility.
"Aligned" foundation

...

Max H — AI Alignment Forum

Top postsTop post

Bayesian updating in real life is mostly about understanding your hypotheses

My sense is that an increasingly common viewpoint around here is that the last ~20 years of AI development and AI x-risk discourse are well-described by the following narrative: > Eliezer Yudkowsky (and various others who were at least initially heavily influenced by his ideas) developed detailed models of key issues likely to be inherent in the process of developing smarter-than-human AI. > These models were somewhere between "maybe plausible" and "quite compelling" at the time that they were put forth, but recent developments in AI (e.g. behavioral characteristics of language models, smoothness / gradualness of scaling) have shown that reality just isn't panning out in quite the way Eliezer's models predicted. > These developments haven't entirely falsified Eliezer's models and key predictions, but there are now plenty of alternative models and theories. Some or all of these competing models either are or claim to: > > * have a better recent track record of predicting near-term AI developments > * better retrodict past developments[1] > * be backed by empirical results in machine learning and / or neuroscience > * feel more intuitively plausible and evidence-backed to people with different backgrounds and areas of expertise > Therefore, even if we can't entirely discount Eliezer's models, there's clearly a directional Bayesian update which any good Bayesian (including Eliezer himself) should be able to make by observing recent developments and considering alternate theories which they support. Even if the precise degree of the overall update (and the final landing place of the posterior) remains highly uncertain and debatable, the basic direction is clear. Without getting into the object-level too much, or even whether the narrative as a whole reflects the actual views of particular real people, I want to make some remarks on the concept of belief updating as typically used in narratives like this. Note, there's a sense in which any (valid) change in on

A decade of lurking, a month of posting

Forum Karma: view stats and find highly-rated comments for any LW user

Steering systems

Reward is the optimization target (of capabilities researchers)

In Reward is not the optimization target, @TurnTrout writes: > Therefore, reward is not the optimization target in two senses: > > 1. Deep reinforcement learning agents will not come to intrinsically and primarily value their reward signal; reward is not the trained agent’s optimization target. > 2. Utility functions...

May 15, 2023•32

Paying the corrigibility tax

In my recent post on steering systems, I sketched an AI system which chooses which actions to execute via tree search over world states. In the original post, the system is meant to illustrate the possibility that component subsystems which are human-level and (possibly) non-dangerous when used individually can be...

Apr 19, 2023•14

"Aligned" foundation models don't imply aligned systems

[The basic idea in this post is probably not original to me, since it's somewhat obvious when stated directly. But it seems particularly relevant and worth distilling in light of recent developments with LLM-based systems, and because I keep seeing arguments which seem confused about it.] Alignment is a property...

Apr 13, 2023•39