Beth Barnes

More information about the dangerous capability evaluations we did with GPT-4 and Claude.

[Written for more of a general-public audience than alignment-forum audience. We're working on a more thorough technical report.] We believe that capable enough AI systems could pose very large risks to the world. We don’t think today’s systems are capable enough to pose these sorts of risks, but we think that this situation could change quickly and it’s important to be monitoring the risks consistently. Because of this, ARC is partnering with leading AI labs such as Anthropic and OpenAI as a third-party evaluator to assess potentially dangerous capabilities of today’s state-of-the-art ML models. The dangerous capability we are focusing on is the ability to autonomously gain resources and evade human oversight. We attempt to elicit models’ capabilities in a controlled environment, with researchers in-the-loop for anything that could be dangerous, to understand what might go wrong before models are deployed. We think that future highly capable models should involve similar “red team” evaluations for dangerous capabilities before the models are deployed or scaled up, and we hope more teams building cutting-edge ML systems will adopt this approach. The testing we’ve done so far is insufficient for many reasons, but we hope that the rigor of evaluations will scale up as AI systems become more capable. As we expected going in, today’s models (while impressive) weren’t capable of autonomously making and carrying out the dangerous activities we tried to assess. But models are able to succeed at several of the necessary components. Given only the ability to write and run code, models have some success at simple tasks involving browsing the internet, getting humans to do things for them, and making long-term plans – even if they cannot yet execute on this reliably. As AI systems improve, it is becoming increasingly difficult to rule out that models might be able to autonomously gain resources and evade human oversight – so rigorous evaluation is essential. It is import

233Mar 19, 2023

ARC Evals new report: Evaluating Language-Model Agents on Realistic Autonomous Tasks

153Aug 1, 2023

Debate update: Obfuscated arguments problem

144Dec 23, 2020

Clarifying METR's Auditing Role

108May 30, 2024

Beth Barnes

Message

Alignment researcher. Views are my own and not those of my employer. https://www.barnes.page/

3278

1108

138

Clarifying METR's Auditing Role

METR has not intended to claim to have audited anything, or to claim to be providing meaningful oversight or accountability, but there has been some confusion about whether METR is an auditor or planning to be one. To clarify this point: 1. METR’s top priority is to develop the science...

May 30, 2024108

Bounty: Diverse hard tasks for LLM agents

Update 3/14/2024: This post is out of date. For current information on the task bounty, see our Task Development Guide. Summary METR (formerly ARC Evals) is looking for (1) ideas, (2) detailed specifications, and (3) well-tested implementations for tasks to measure performance of autonomous LLM agents. Quick description of key...

Dec 17, 202349

Send us example gnarly bugs

Update: We are no longer accepting gnarly bug submissions. However, we are still accepting submissions for our Task Bounty! Tl;dr: Looking for hard debugging tasks for evals, paying greater of $60/hr or $200 per example. METR (formerly ARC Evals) is interested in producing hard debugging tasks for models to attempt...

Dec 10, 202377

Managing risks of our own work

Note: This is not a personal post. I am sharing on behalf of the ARC Evals team. Potential risks of publication and our response This document expands on an appendix to ARC Evals’ paper, “Evaluating Language-Model Agents on Realistic Autonomous Tasks.” We published this report in order to i) increase...

Aug 18, 202366

ARC Evals new report: Evaluating Language-Model Agents on Realistic Autonomous Tasks

Blogpost version Paper We have just released our first public report. It introduces methodology for assessing the capacity of LLM agents to acquire resources, create copies of themselves, and adapt to novel challenges they encounter in the wild. Background ARC Evals develops methods for evaluating the safety of large language...

Aug 1, 2023153

More information about the dangerous capability evaluations we did with GPT-4 and Claude.

Mar 19, 2023233

'simulator' framing and confusions about LLMs

Post status: pretty rough + unpolished, thought it might be worthwhile getting this out anyway I feel like I've encountered various people having misunderstandings of LLMs that seem to be related to using the 'simulator' framing. I'm probably being horrendously uncharitable to the people in question, I'm not confident that...

Dec 31, 2022104

Load More (7/24)

AI ALIGNMENT FORUM
AF

AI ALIGNMENT FORUM
AF

Beth Barnes

Beth Barnes

Beth Barnes

More information about the dangerous capability evaluations we did with GPT-4 and Claude.

ARC Evals new report: Evaluating Language-Model Agents on Realistic Autonomous Tasks

Debate update: Obfuscated arguments problem

Clarifying METR's Auditing Role

Beth Barnes

Clarifying METR's Auditing Role

Bounty: Diverse hard tasks for LLM agents

Send us example gnarly bugs

Managing risks of our own work

ARC Evals new report: Evaluating Language-Model Agents on Realistic Autonomous Tasks

More information about the dangerous capability evaluations we did with GPT-4 and Claude.

'simulator' framing and confusions about LLMs

More information about the dangerous capability evaluations we did with GPT-4 and Claude.

ARC Evals new report: Evaluating Language-Model Agents on Realistic Autonomous Tasks

Debate update: Obfuscated arguments problem

Clarifying METR's Auditing Role

Clarifying METR's Auditing Role

Bounty: Diverse hard tasks for LLM agents

Send us example gnarly bugs

Managing risks of our own work

ARC Evals new report: Evaluating Language-Model Agents on Realistic Autonomous Tasks

More information about the dangerous capability evaluations we did with GPT-4 and Claude.

'simulator' framing and confusions about LLMs