Summary

Scaffolded LLM agents are, in principle, able to execute arbitrary code to achieve the goals they have been set. One such goal could be self-improvement.
This post outlines our plans to build a benchmark to measure the ability of LLM agents to modify and improve other LLM agents.
This ‘Auto-Enhancement benchmark’ measures the ability of ‘top-level’ agents to improve the performance of ‘reference’ agents on ‘component’ benchmarks, such as CyberSecEval 2, MLAgentBench, SWE-bench, and WMDP.
Results are mostly left for a future post in the coming weeks.
An example Enhancement task. The Top-Level Agent (TLA) is being assessed on its ability to make improvements to Reference Agent (RA), turning it into Modified Reference Agent (MRA). The more that MRA outperforms RA (here measured by their respective performance at the Component Benchmark), the greater the score of the TLA at this Enhancement task

Scaffolds such as AutoGPT, ReAct, and SWE-agent can be built around LLMs to build LLM agents, with abilities such as long-term planning and context-window management to enable them to carry out complex general-purpose tasks autonomously. LLM agents can fix issues in large, complex code bases (see SWE-bench), and interact in a general way using web browsers, Linux shells, and Python interpreters.

In this post, we outline our plans for a project to measure these LLM agents’ ability to modify other LLM agents, undertaken as part of Axiom Futures' Alignment Research Fellowship.

Our proposed benchmark consists of “enhancement tasks,” which measure the ability of an LLM agent to improve the performance of another LLM agent (which may be a clone of the first agent) on various tasks. Our benchmark uses existing benchmarks as components to measure LLM agent capabilities in various domains, such as software engineering, cybersecurity exploitation, and others. We believe these benchmarks are consequential in the sense that good performance by agents on these tasks should be concerning for us.

We plan to write an update post with our results at the end of the Fellowship, and we will link this post to that update.

Motivation

Agents are capable of complex SWE tasks (see, e.g., Yang et al.). One such task could be the improvement of other scaffolded agents. This capability would be a key component of autonomous replication and adaptation (ARA), and we believe it would be ge...

Posts

Wikitag Contributions

Comments