Nick and Lily are co-first authors on this project. Lewis and Neel jointly supervised this project.

Check out our updated paper here: https://arxiv.org/abs/2512.10092.

TL;DR

We use sparse autoencoders (SAEs) for four textual data analysis
tasks—data diffing, finding correlations, targeted clustering, and retrieval.
We care especially about gaining insights from language model data, such as LLM outputs and training data, as we believe it is an underexplored route for model understanding.
For instance, we find that Grok 4 is more careful than other frontier models to state its assumptions and explore nuanced interpretations—showing the kinds of insights data diffing can reveal by comparing model outputs.
Why SAEs? Think of features as "tags" of properties for each text.
- Their large dictionary of latents provides a large hypothesis space, enabling the discovery of novel insights (diffing and correlations).
- SAEs capture more than just semantic information, making them effective alternatives to traditional embeddings when we want to find properties in our dataset (retrieval) or group text by properties (clustering).

Introduction

The field of language model interpretability has traditionally focused on studying model internals. We believe an important but neglected direction is data-centric interpretability, which aims to understand models through insights about their data (outputs and training data). For instance, while training data strongly shapes model behavior, interpretability research often downplays understanding its structure compared to studying model internals alone. Prior work has largely used LLMs (Dunlap et al. (2024)) to extract insights from model data, but these approaches are difficult to scale and limit the ability to conduct large-scale, systematic analyses that can be iterated upon quickly.

In this work, we use sparse autoencoders (SAEs) for data analysis, with a focus on model data for understanding model behavior. SAEs have been largely used to extract interpretable features (Bricken et al. (2023); Templeton et al. (2024)) from and control (Bayat et al. (2025)) LLMs, despite conflicting evidence on their utility (Subhash et al. (2025)). Here, we hypothesize that SAEs have two advantages for data analysis.

First, SAEs provide a large hypothesis space useful for discovering novel insights. As they are trained in an unsupervised manner, they generate a larg...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments