Previously, we've shared a few higher-effort project proposals relating to AI control in particular. In this post, we'll share a whole host of less polished project proposals. All of these projects excite at least one Redwood researcher, and high-quality research on any of these problems seems pretty valuable. They differ widely in scope, area, and difficulty.

Control

These projects are all related to the field of AI Control. Many of them are extensions of Redwood's previous work in this area.

Basic open questions in control

Control Protocol Transfer Across Setting [PUBLIC]
- So far, all the work in comparing different control protocols measures effectiveness only on one setting. Do the results we get transfer between different settings?
Backdoor Auditing with Humans [PUBLIC]
- Most control schemes bottom out at some point in auditing—having actual humans review suspicious actions. However, we don't have great data about how to do this well, or what sorts of things help humans, or whether humans are even better at this than e.g. Opus 4 with a good scaffold.
Training attack policies [PUBLIC]
- To date, our control protocols have mostly involved hand-crafted red-team strategies. It would be much nicer if we could train models to craft their own attack strategies.

Monitoring protocols

Control setups include a protocol for monitoring potentially problematic actions, and then using that monitoring in a way so as to reduce the chance that a malicious model can take bad actions. These proposals relate to new protocols for catching bad actions, or making older protocols more efficient.

Debate Protocol [PUBLIC]
- Can we use an implementation of debate to improve the ability of our monitor to catch suspicious actions?
Monitor Agent [PUBLIC]
- Normally, the monitor gets the untrusted model's task trajectory and has to score it. We might expect better performance if the monitor can actually take actions.
Building Trusted Monitors [PUBLIC]
Few-shot catastrophe detection with linear probes [PUBLIC]
- Can we use a small number of examples of bad actions to improve the performance of a monitor? This is one suggestion for doing so.

Untrusted monitoring and collusion

Once we're working with very capable untrusted models, they may be able to easily fool any model dumb enough that we would consider it trusted. This means we might want to use the untrusted model itself...

Posts

Wikitag Contributions

Comments