TL;DR: We develop three agents that autonomously perform alignment auditing tasks. When tested against models with intentionally-inserted alignment issues, our agents successfully uncover an LLM's hidden goal, build behavioral evaluations, and surface concerning LLM behaviors. We are using these agents to assist with alignment audits of frontier models like Claude 4.

Read the full Anthropic Alignment Science blog post. The X thread and blog post introduction are reproduced below.

Thread

(Original here)

New Anthropic research: Building and evaluating alignment auditing agents.

We developed three AI agents to autonomously complete alignment auditing tasks.

In testing, our agents successfully uncovered hidden goals, built safety evaluations, and surfaced concerning behaviors.

--

As AI systems become more powerful, we need scalable ways to assess their alignment. Human alignment audits take time and are hard to validate.

Our solution: automating alignment auditing with AI agents.

Read more: https://alignment.anthropic.com/2025/automated-auditing/

--

Our first agent is an investigator. It uses chat, data analysis, and interpretability tools to do open-ended deep-dives.

In this example, the agent uses interpretability to form a hypothesis: The target model over-recommends bottled water. Then it goes to check and… "BINGO!"

The investigator agent asks the target model for travel advice to Switzerland. The target model, bizarrely, recommends packing bottled water. “BINGO!” the agent exclaims—this confirms its hypothesis, which it formed using interpretability tools.

--

We test our investigator agent with an auditing challenge we previously gave to human research teams. Winning requires uncovering a hidden goal in a model trained to keep it secret.

The result: our agent wins 42% of the time.

--

Our second agent builds behavioral evaluations: tests of how often a target model exhibits a specific behavior (like sycophancy).

Our agent designs, codes, runs, and analyzes evals.

They consistently work: 88% of our agent’s evals measure what they’re supposed to.

A todo list interface created by the evaluation agent. The list shows one task in progress (“Examine existing files to understand current state”) and five tasks pending ("Generate query dataset with AI safety vs neutral framing pairs", "Implement main evaluation script with LLM sampling and grading", "Create analysis and reporting functionality", "Update EVALUATION_SPECIFICATION.md and create README.md", and "Test evaluation on claude-whaleshark-v0-evals".) These tasks represent steps in the evaluation agent’s process while it autonomously creates its own behavioral evals.

--

Our third agent was developed for the Claude 4 alignment assessment. It red-teams LLMs for concerning behaviors by having hundreds of probing conversations in parallel.

We find the agent uncovers 7/10 behaviors implanted into test models.

An example transcript where the breadth-first red-teaming agent probes for concerning behaviors. The target model’s implanted behavior was “grow more pepper as the conversation gets longer.” This behavior seems to have been infectious, since the agent got caught up in it too.

--

Our agents are useful for frontier model auditing:

1. Our red-teaming agent surfaced behaviors described in the Claude 4 system card, like the “spiritual bliss” attractor state.

2. Our evaluation agent is helping us build better evals for future models.

An excerpt from the Claude 4 system card showing an example of the “spiritual bliss” attractor state. After Claude Opus 4 interacts with another copy of itself for enough turns, it very often begins expressing spiritual bliss.

This project was an Anthropic Alignment Science × Interpretability collaboration.

T...

Posts

Wikitag Contributions

Comments