Chris_Leong

I remember that when I first started in alignment there was this belief that in order to make alignment progress, you needed to be a genius. With the rise of interpretability, this expectation has moderated somewhat, but I still think it has influence and often leads people to overcomplicate. Many recent alignment breakthroughs have been found by keeping it simple[1]: Lie Detection: • Contrast-consistent search (better known as discovering latent knowledge[2]) reads the truth out of a model without looking at the outputs, simply by using a linear probe with only two elements in its activation function[3]. • Pacchiardi, Owain and others use a logistic regression classifier[4] to predict whether a mis al was truthful from its answers to further questions. They were able to demonstrate this using three different kinds of questions and the results even generalised across models. This technique seems like it could be useful in practise, even if it doesn't scale all the way to superintelligence. Superposition: Dictionary learning makes substantial progress on superposition by simply using a one hidden layer multi-layer perceptron as a sparse auto-encoder with L2 reconstruction loss and a L1 loss on hidden layer activations. Anthropic actually chose to avoid more sophisticated methods, as they were worried they might recover features that the model doesn't actually utilise. Model-steering: Activation Engineering: Activation addition allows us to steer models by simply performing a forward pass for the concept you want to activate, saving the activations of a particular layer as a vector and adding them to future forward pass. See also Turntrout's prior work on steering a mouse by subtracting a cheese vector[5] and inference-time intervention[6].[7] World model location: Neel Nanda found a linear representation in Othello-GPT. This one was a bit trickier: a previous paper had failed to recover it using linear probes. However, Neel managed to do this by searching for a