Satvik Golechha

Message

I research intelligence and it’s emergence and expression in neural networks to ensure advanced AI is safe and beneficial.

Current interests: neural network interpretability, alignment/safety, unsupervised learning, and deep learning theory.

For more, check out my scholar profile and personal website.

384

Satvik Golechha

Auditing language models for hidden objectives

We study alignment audits—systematic investigations into whether an AI is pursuing hidden objectives—by training a model with a hidden misaligned objective and asking teams of blinded researchers to investigate it. This paper was a collaboration between the Anthropic Alignment Science and Interpretability teams. Abstract We study the feasibility of conducting...

Mar 13, 2025145

We study alignment audits—systematic investigations into whether an AI is pursuing hidden objectives—by training a model with a hidden misaligned objective and asking teams of blinded researchers to investigate it.

This paper was a collaboration between the Anthropic Alignment Science and Interpretability teams.

Abstract

We study the feasibility of conducting alignment audits: investigations into whether models have undesired objectives. As a testbed, we train a language model with a hidden objective. Our training pipeline first teaches the model about exploitable errors in RLHF reward models (RMs), then trains the model to exploit some of these errors. We verify via out-of-distribution evaluations that the model generalizes to exhibit whatever behaviors it believes RMs rate highly, including... (read 3642 more words →)

AI ALIGNMENT FORUM
AF

AI ALIGNMENT FORUM
AF

Satvik Golechha

Satvik Golechha

Satvik Golechha

Auditing language models for hidden objectives

Satvik Golechha

Satvik Golechha

Satvik Golechha

Auditing language models for hidden objectives

Abstract