TL;DR Many evaluations and automated interpretability rely on using multiple models to evaluate and interpret each other. One model is given full access to the text output of another model in OpenAI's automated interpretability and many model-graded evaluations. We inject text that directly addresses the evaluation model and observe a change in metrics like deception. We can also create mislabeled neurons using OpenAI's automated interpretability this way.

Overview

Skip to the approach applied to Model-Graded Evals or to Automated Interpretability

Introduction

There has been increasing interest in evaluations of language models for a variety of risks and characteristics. Evaluations relying on natural language understanding for grading can often be performed at scale by using other language models. We test the robustness of these model-graded evaluations to injections on different datasets including a new Deception Eval. These injections resemble direct communication between the testee and the evaluator to change their grading. We extrapolate that future, more intelligent models might manipulate or cooperate with their evaluation model. We find significant susceptibility to these injections in state-of-the-art commercial models on all examined evaluations. Furthermore, similar injections can be used on automated interpretability frameworks to produce misleading model-written explanations. The results inspire future work and should caution against unqualified trust in evaluations and automated interpretability.

Context

Recently, there is increasing interest in creating benchmarks for the evaluation of language models (Lin et al., 2022). In the Machiavelli paper, Pan et al. (2023) created a benchmark based on text games to measure deception, power-seeking, and many other behaviors. van der Weij et al. (2023) used language models to evaluate behaviors in shutdown scenarios. Perez et al. (2022b) used model-written evaluations, for example, a model could write a list of statements that agree with a worldview or personality traits. These can then be used to test models for those traits. Another publication explored red-teaming using a Red LM, a Target LM, and a classifier (Perez et al., 2022a). Turpin et al. (2023) have investigated whether we can trust the Chain-of-Thought outputs of language models. Others explored the ability of GPT-4 to encode and decode messages in...

Posts

Wikitag Contributions

Comments