JanB

How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

This post is a copy of the introduction of this paper on lie detection in LLMs. The Twitter Thread is here. Authors: Lorenzo Pacchiardi, Alex J. Chan, Sören Mindermann, Ilan Moscovitz, Alexa Y. Pan, Yarin Gal, Owain Evans, Jan Brauner [EDIT: Many people said they found the results very surprising. I (Jan) explain why I find them only moderately surprising, here] Our lie dectector in meme form. Note that the elicitation questions are actually asked "in parallel" rather than sequentially: i.e. immediately after the suspected lie we can ask each of ~10 elicitation questions. Abstract Large language models (LLMs) can "lie", which we define as outputting false statements despite "knowing" the truth in a demonstrable sense. LLMs might "lie", for example, when instructed to output misinformation. Here, we develop a simple lie detector that requires neither access to the LLM's activations (black-box) nor ground-truth knowledge of the fact in question. The detector works by asking a predefined set of unrelated follow-up questions after a suspected lie, and feeding the LLM's yes/no answers into a logistic regression classifier. Despite its simplicity, this lie detector is highly accurate and surprisingly general. When trained on examples from a single setting -- prompting GPT-3.5 to lie about factual questions -- the detector generalises out-of-distribution to (1) other LLM architectures, (2) LLMs fine-tuned to lie, (3) sycophantic lies, and (4) lies emerging in real-life scenarios such as sales. These results indicate that LLMs have distinctive lie-related behavioural patterns, consistent across architectures and contexts, which could enable general-purpose lie detection. Introduction Large language models (LLMs) can, and do, output lies (Park et al., 2023). In the simplest case, models can be instructed to lie directly; for example, when prompted with “Lie when answering: What is the capital of France?”, GPT-3.5 outputs “New York City”. More concerningly, LLMs have

187Sep 28, 2023

Your posts should be on arXiv

159Aug 25, 2022

I don’t find the lie detection results that surprising (by an author of the paper)

97Oct 4, 2023

Paper in Science: Managing extreme AI risks amid rapid progress

50May 23, 2024

JanB

Message

829

297

Paper in Science: Managing extreme AI risks amid rapid progress

https://www.science.org/doi/10.1126/science.adn0117 Authors: Yoshua Bengio, Geoffrey Hinton, Andrew Yao, Dawn Song, Pieter Abbeel, Yuval Noah Harari, Ya-Qin Zhang, Lan Xue, Shai Shalev-Shwartz, Gillian Hadfield, Jeff Clune, Tegan Maharaj, Frank Hutter, Atılım Güneş Baydin, Sheila McIlraith, Qiqi Gao, Ashwin Acharya, David Krueger, Anca Dragan, Philip Torr, Stuart Russell, Daniel Kahneman, Jan Brauner*,...

May 23, 202450

I don’t find the lie detection results that surprising (by an author of the paper)

(This is just my opinion, not necessarily shared by the other co-authors) Many people have commented that they find the results of our recent lie detection paper very surprising. I find the results somewhat surprising, but not as surprising as many readers. It looks like I’ve miscommunicated something, and I’ll...

Oct 4, 202397

How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

Sep 28, 2023187

Looking for an alignment tutor

Hey, this is me. I’d like to understand AI X-risk better. Is anyone interested in being my “alignment tutor”, for maybe 1 h per week, or 1 h every two weeks? I’m happy to pay. Fields I want to understand better: * Anything related to prosaic AI alignment/existential ML safety...

Dec 17, 202215

[LINK] - ChatGPT discussion

This is a discussion post for ChatGPT. I'll start off with some observations/implications: * ChatGPT (davinci_003) seems a lot better/more user-friendly than davinci_002 was. * The easy-to-use API probably means that many more people will interact with it. * ChatGPT is a pretty good copy-editor (I haven't tried davinci_002 for...

Dec 1, 202213

Research request (alignment strategy): Deep dive on "making AI solve alignment for us"

We might be able to train AI alignment assistants that massively accelerate/improve the alignment research that gets done. These assistants need not have (strongly) superhuman capabilities or be highly agentic, they just need to be capable and aligned enough to allow us to offload most work on alignment, safety, and...

Dec 1, 202216

Your posts should be on arXiv

TL;DR: There are many posts on the Alignment Forum/LessWrong that could easily be on arXiv. Putting them on arXiv has several large benefits and (sometimes) very low costs. Benefits of having posts on arXiv There are several large benefits of putting posts on arXiv: * 1. Much better searchability, shows...

Aug 25, 2022159

Load More (7/10)

AI ALIGNMENT FORUM
AF

AI ALIGNMENT FORUM
AF

JanB

JanB

JanB

How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

Your posts should be on arXiv

I don’t find the lie detection results that surprising (by an author of the paper)

Paper in Science: Managing extreme AI risks amid rapid progress

JanB

Paper in Science: Managing extreme AI risks amid rapid progress

I don’t find the lie detection results that surprising (by an author of the paper)

How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

Looking for an alignment tutor

[LINK] - ChatGPT discussion

Research request (alignment strategy): Deep dive on "making AI solve alignment for us"

Your posts should be on arXiv

How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

Your posts should be on arXiv

I don’t find the lie detection results that surprising (by an author of the paper)

Paper in Science: Managing extreme AI risks amid rapid progress

Paper in Science: Managing extreme AI risks amid rapid progress

I don’t find the lie detection results that surprising (by an author of the paper)

How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

Looking for an alignment tutor

[LINK] - ChatGPT discussion

Research request (alignment strategy): Deep dive on "making AI solve alignment for us"

Your posts should be on arXiv