The Alignment Project is a global fund of over £15 million, dedicated to accelerating progress in AI control and alignment research. It is backed by an international coalition of governments, industry, venture capital and philanthropic funders.

This sequence sets out the research areas we are excited to fund – we hope this list of research ideas presents a novel contribution to the alignment field. We have deliberately focused on areas that we think the AI safety community currently underrates.

Apply now to join researchers worldwide in advancing AI safety.

For those with experience scaling and running ambitious projects, apply to our Strategy & Operations role here.

Our research goals

In-scope projects will aim to address either of the following challenges:

AI Control: How can we prevent AI systems from carrying out actions that pose risks to our collective security, even when they may attempt to carry out such actions?
AI Alignment: How can we design AI systems which do not attempt to carry out such actions in the first place?

Making substantial breakthroughs in these areas is an interdisciplinary effort, requiring a diversity of tools and perspectives. We want the best and the brightest across many fields to contribute to alignment research, so have organised these priority research areas as a set of discipline-specific questions. We suggest clicking ahead to your specific areas of interest, rather than reading linearly. Sections are roughly ordered from most theoretical to most empirical.

Some of the subfields below have more detail than others about subproblems, recent work, and related work. This should not be read as a signal about which areas we believe are more important: much of the variance is due to areas our alignment and control teams, or our collaborators, have focused on to date. So, for example, lots of the alignment questions focus on scalable oversight / debate. We want to bring other areas up to similar levels of detail, and will attempt to do this in future versions of this agenda.

We’re excited about funding projects that tackle these questions, even if they aren’t focused on a problem outlined below. Feel free to look at others’ lists and overviews — e.g. Google DeepMind, Anthropic, or Redwood Research — for ideas. If you see connections between your research and these challenges, we enc...

Posts

Wikitag Contributions

Comments