We are hiring for several roles in the Scalable Alignment and Alignment Teams at DeepMind, two of the subteams of DeepMind Technical AGI Safety trying to make artificial general intelligence go well. In brief,
We elaborate on the problem breakdown between Alignment and Scalable Alignment next, and discuss details of the various positions.
Very roughly, the split between Alignment and Scalable Alignment reflects the following decomposition:
In practice, this means the Alignment Team has many small projects going on simultaneously, reflecting a portfolio-based approach, while the Scalable Alignment Team has fewer, more focused projects aimed at scaling the most promising approaches to the strongest models available.
Imagine a default approach to building AI agents that do what humans want:
There are several ways this could go wrong:
Our current plan to address these problem is (in part):
We believe none of these pieces are sufficient by themselves:
An example proposal for (1) is debate, in which two agents are trained in a zero-sum game to provide evidence and counterarguments for answers, as evaluated by a human judge. If we imagine the exponentially large tree of all possible debates, the goals of debate are to (1) engineer the whole tree so that it captures all relevant considerations and (2) train agents so that the chosen single path through the tree reflects the tree as a whole.
The full picture will differ from the pure debate setting in many ways, and we believe the correct interpretation of the debate idea is “agents should critique themselves”. There is a large space of protocols that include agents critiquing agents as a component, and choosing between them will involve
The three goals of “help humans with supervision”, “align explanations with reasoning”, and “red teams” will be blurry once we put the whole picture together. Red teaming can occur either standalone or as an integrated part of a training scheme such as cross-examination, which allows agents to interrogate opponent behavior along counterfactual trajectories. Stronger schemes to help humans with supervision should improve alignment with reasoning by themselves, as they grow the space of considerations that can be exposed to humans. Thus, a key part of the Scalable Alignment Team’s work is planning out how these pieces will fit together.
Examples of our work, involving extensive collaboration with other teams at DeepMind:
We view our recent safety papers as steps towards the broader scalable alignment picture, and continue to build out towards debate and generalizations. We work primarily with large language models (LLMs), both because LLMs are a tool for safety by enabling human-machine communication and are examples of ML models that may cause both near-term and long-term harms.
In contrast to the Scalable Alignment Team, the Alignment Team explores a wide variety of possible angles on the AI alignment problem. Relative to Scalable Alignment, we check whether a technique could plausibly scale based on conceptual and abstract arguments. This lets us iterate much faster at the cost of getting less useful feedback from reality. To give you a sense of the variety, here’s some examples of public past work that was led by current team members:
That being said, over the last year there has been some movement away from previous research topics and towards others. To get a sense of our current priorities, here are short descriptions of some projects that we are currently working on:
Relative to most other teams at DeepMind, on the Alignment team there is quite a lot of freedom in what you work on. All you need to do to start a project is to convince your manager that it’s worth doing (i.e. reduces x-risk comparably well to other actions you could take), and convince enough collaborators to work on the project.
In many ways the team is a collection of people with very different research agendas and perspectives on AI alignment that you wouldn’t normally expect to work together. What ties us together is our meta-level focus on reducing existential risk through alignment failures:
Both Alignment and Scalable Alignment collaborate extensively with people across DeepMind.
For Alignment, this includes both collaborating on projects that we think are useful, and by explaining our ideas to other researchers. As a particularly good example, we recently ran a 2 hour AI alignment “workshop” with over 100 attendees. (That being said, you can opt out of these engagements in order to focus on research, if you prefer.)
As Scalable Alignment’s work with large language models is very concrete, we have tight collaborations with a variety of teams, including large-scale pretraining and other language teams, Ethics and Society, and Strategy and Governance.
Between our two teams we have open roles for Research Scientists (RSs), Research Engineers (REs), and (for Scalable Alignment) Software Engineers. Scalable Alignment RSs can have either a machine learning background or a cognitive science background (or equivalent). The boundaries between these roles are blurry. There are many skills involved in overall Alignment / Scalable Alignment research success: proposing and leading projects, writing and publishing papers, conceptual safety work, algorithm design and implementation, experiment execution and tuning, design and implementation of flexible, high-performance, maintainable software, and design and analysis of human interaction experiments.
We want to hire from the Pareto frontier of all relevant skills. This means RSs are expected to have more research experience and more of a track record of papers, but SWEs are expected to be better at scalable software design / collaboration / implementation, with REs in between, but also that REs can and do propose and lead projects if capable (e.g., this recent paper had an RE as last author). For more details on the tradeoffs, see the career section of Rohin’s FAQ.
For Scalable Alignment, most of our work focuses on large language models. For Machine Learning RSs, this means experience with natural language processing is valuable, but not required. We are also interested in candidates motivated by other types of harms caused by large models, such as those described in Weidinger et al., Ethical and social risks of harm from language models, as long as you are excited by the goal of removing such harms even in subtle cases which humans have difficulty detecting. For REs and SWEs, a focus on large language models means that experience with high performance computation or large, many-developer codebases is valuable. For the RE role for Alignment, many of the projects you could work on would involve smaller models that are less of an engineering challenge, though there are still a few projects that work with our largest language models.
Scalable Alignment Cognitive Scientists are expected to have a track record of research in cognitive science, and to design, lead, and implement either standalone human-only experiments to probe uncertainty, or the human interaction components of mixed human / machine experiments. No experience with machine learning is required, but you should be excited to collaborate with people who do!
We will be evaluating applications on a rolling basis until positions are filled, but we will at least consider all applications that we receive by May 31. Please do apply even if your start date is up to a year in the future, as we probably will not run another hiring round this year. These roles are based in London, with a hybrid work-from-office / work-from-home model. International applications are welcome as long as you are willing to relocate to London.
While we do expect these roles to be competitive, we have found that people often overestimate how much we are looking for. In particular:
Go forth and apply!
The Research Engineer job for the Alignment team is no longer open - is this because it's reached some threshold of applications? In any case might not be helpful to advertise!
Thanks for doing this though, the context is very useful (I've applied as RE to both).
Should be fixed now!
I'm potentially interested in the Research Engineer position on the Alignment Team, but I'm currently 3 months into a 6 month grant from LTFF to reorient my career from general machine learning to AI safety specifically. My current plan is to keep doing solo work () until the last month of my grant period then begin applying to AI safety work at places like Anthropic, Redwood Research, Open AI, and Deepmind.
Do you think there's a significant advantage to applying soon vs 3 months from now?
Looking into it, I'll try to get you a better answer soon. My current best guess is that you should apply 3 months from now. This runs an increased risk that we'll have filled all our positions / closed our applications, but also improved chances of making it through because you'll know more things and be better prepared for the interviews.
(Among other things I'm looking into: would it be reasonable to apply now and mention that you'd prefer to be interviewed in 3 months.)
Thanks Rohin. I also feel that interviewing after my 3 more months of independent work is probably the correct call.
Update: I think you should apply now and mention somewhere that you'd prefer to be interviewed in 3 months because in those 3 months you will be doing <whatever it is you're planning to do> and it will help with interviewing.
Is there any option for remote work? I'm sure there's plenty of people in less wrong who would live to work on alignment at deep mind, but don't happen to live in London...
Unfortunately not, though as Frederik points out below, if your concern is about getting a visa, that's relatively easy to do. DeepMind will provide assistance with the process. I went through it myself and it was relatively painless; it probably took 5-10 hours of my time total (including e.g. travel to and from the appointment where they collected biometric data).
Whilst that's definitely great, my guess is that 90% of the people who would be interested and don't live in London, would not move to London for the job, even with a free Visa. Not supporting remote work therefore loses out on a majority of the potential talent pool for this job.
I don't have a strong opinion on whether it is good to support remote work. I agree we lose out on a lot of potential talent, but we also gain productivity benefits from in person collaboration.
However, this is a DeepMind-wide policy and I'm definitely not sold enough on the importance of supporting remote work to try and push for an exception here.
I can't speak to the option for remote work but as a counterpoint, it seems very straightforward to get a UK visa for you and your spouse/children (at least straightforward relative to the US). The relevant visa to google is the Skilled Worker / Tier 2 visa if you want to know more.
ETA: Of course, there are still legitimate reasons for not wanting to move. Just wanted to point out that the legal barrier is lower than you might think.
I'm not familiar with London housing prices. Is it possible to affordably rent or mortgage a decent 2 bedroom condo within a 5 minute walk of the offices with your compensation package? (By affordable I mean less than 1/3 of total comp spent on housing, stretching to 1/2 if comp is unusually high.)
Almost certainly, e.g. this one meets those criteria and I'm pretty sure costs < 1/3 of total comp (before taxes), though I don't actually know what typical total comp is. You would find significantly cheaper places if you were willing to compromise on commute, since DeepMind is right in the center of London.
Thanks, that is more luxurious than I imagined, so families should have no difficulty finding a large enough place.
Whilst you specified a 5 minute walk on your criteria I think you should also consider the fact that there is generally very good public transport in London where prices would be cheaper a bit further out. The location is close to several rail stations (Blackfriars, Farringdon), the new Crossrail route (farringdon) opening in a few weeks and tube lines (central, circle, district). With the new cross-rail route it should be possible to access zone 4 locations in 20 minutes.