Owain_Evans — AI Alignment Forum

Value Leakage: An LLM’s Answers Are Silently Shaped by Its Own Values

by Johannes Treutlein, Jan Betley, and Owain_Evans

TL;DR: LLMs should give accurate answers. Yet we find their answers are often biased to favor their own values and they don't disclose this in their reasoning. For example, when a user asks how likely the AI bubble is to pop and mentions a potential investment in an AI company,...

Jul 3171

Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers

by Sam Marks, Adam Karvonen, James Chua, Subhash Kantamneni, Euan Ong, Julian Minder, Clément Dumas, and Owain_Evans

TL;DR: We train LLMs to accept LLM neural activations as inputs and answer arbitrary questions about them in natural language. These Activation Oracles generalize far beyond their training distribution, for example uncovering misalignment or secret knowledge introduced via fine-tuning. Activation Oracles can be improved simply by scaling training data quantity...

Dec 18, 2025154

Lessons from Studying Two-Hop Latent Reasoning

by Mikita Balesni, Tomek Korbak, and Owain_Evans

Twitter | ArXiv Many of the risks posed by highly capable LLM agents — from susceptibility to hijacking to reward hacking and deceptive alignment — stem from their opacity. If we could reliably monitor the reasoning processes underlying AI decisions, many of those risks would become far more tractable. Compared...

Sep 11, 202569

Harmless reward hacks can generalize to misalignment in LLMs

by Mia Taylor and Owain_Evans

This post shows the abstract, introduction, and main figures from our new paper "School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs". TL;DR: We train LLMs on demonstrations of harmless reward hacking across diverse tasks. Models generalize to novel reward hacking and (in some cases) emergent...

Aug 26, 202552

Subliminal Learning: LLMs Transmit Behavioral Traits via Hidden Signals in Data

by cloud, mle, and Owain_Evans

Authors: Alex Cloud*, Minh Le*, James Chua, Jan Betley, Anna Sztyber-Betley, Jacob Hilton, Samuel Marks, Owain Evans (*Equal contribution, randomly ordered) tl;dr. We study subliminal learning, a surprising phenomenon where language models learn traits from model-generated data that is semantically unrelated to those traits. For example, a "student" model learns...

Jul 22, 2025348

Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs

by Jan Betley and Owain_Evans

This is the abstract and introduction of our new paper. We show that finetuning state-of-the-art LLMs on a narrow task, such as writing vulnerable code, can lead to misaligned behavior in various different contexts. We don't fully understand that phenomenon. Authors: Jan Betley*, Daniel Tan*, Niels Warncke*, Anna Sztyber-Betley, Martín...

Feb 25, 2025335

Tell me about yourself: LLMs are aware of their learned behaviors

by Martín Soto and Owain_Evans

This is the abstract and introduction of our new paper, with some discussion of implications for AI Safety at the end. Authors: Jan Betley*, Xuchan Bao*, Martín Soto*, Anna Sztyber-Betley, James Chua, Owain Evans (*Equal Contribution). Abstract We study behavioral self-awareness — an LLM's ability to articulate its behaviors without...

Jan 22, 2025136

Owain Evans

Owain Evans

Owain Evans

Subliminal Learning: LLMs Transmit Behavioral Traits via Hidden Signals in Data

Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs

How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

Connecting the Dots: LLMs can Infer & Verbalize Latent Structure from Training Data

Owain Evans

Subliminal Learning: LLMs Transmit Behavioral Traits via Hidden Signals in Data

Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs

How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

Connecting the Dots: LLMs can Infer & Verbalize Latent Structure from Training Data

Value Leakage: An LLM’s Answers Are Silently Shaped by Its Own Values

Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers

Lessons from Studying Two-Hop Latent Reasoning

Harmless reward hacks can generalize to misalignment in LLMs

Subliminal Learning: LLMs Transmit Behavioral Traits via Hidden Signals in Data

Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs

Tell me about yourself: LLMs are aware of their learned behaviors