Even if the misalignment risk from current AI agents is small, it may be useful to start internally deploying misalignment classifiers: language models designed to classify transcripts that represent intentionally misaligned behaviour. Deploying misalignment classifiers now may provide qualitative and quantitative evidence about how current agents misbehave in real deployments. Further, we can later adapt misalignment classifiers into control monitors: language models that review transcripts with the objective of informing decisions (e.g. whether to block a tool call), in order to mitigate misalignment risks.

Misalignment classifiers and control monitors sound similar, but misalignment classifiers are harder to adversarially evaluate because the concept of an “intentionally misaligned trajectory" is too fuzzy to withstand the optimisation pressure of a red team. This post explains the classifier/monitor distinction, why we think misalignment classifiers are harder to evaluate, why it might be useful to deploy one anyway, our best-guess method for developing them, and some early experimental results.

We would like to thank the broader UK AISI’s Control Team (including Asa Cooper Stickland, Tomek Korbak), Geoffrey Irving and David Lindner who provided significant helpful discussions early-on. Thanks also to Alex Mallen, Fabien Roger, Nate Burnikell, James Peters Gill, Rich BC, and Simon Storf for feedback. 

1. Misalignment classifiers versus control monitors

Let’s say a control monitor is an AI that scores transcripts from an agent (another AI) to inform interventions that reduce the risk of a particular unacceptable outcome. A control monitor’s accuracy in identifying dangerous tool calls is instrumentally useful to reducing risk (e.g. we can replace or audit dangerous tool calls). Similarly, a control monitor’s ability to gather evidence about whether your agent is a schemer could be instrumentally useful to reduce risk (e.g. because we can justify some costly actions, such as undeploying an agent). Regardless: the terminal goal for your control monitor is to reduce the risk posed by the agent.

In contrast, UK AISI’s control team and other researchers have recently explore variants of what we will call (intentional) misalignment classifiers: AIs that classify transcripts from an agent according to whether they attempt an action despite knowing it is in violation of the alignment specification (e.g., Model Spec, Constitution). The core difference between a misalignment classifier and a control monitor is that misalignment classifiers are evaluated on their accurate classification of a behavioural property, but a control monitor can be evaluated by its downstream ability to reduce some concrete consequences when used in some control protocol.^[1] Most of the important parts of this distinction also apply to classifiers for “deployment time scheming” or other properties or “deception”.^[2]

There are two immediate problems one might have with a classifier for “intentionally misaligned trajectories”:

1. First, “intentionally misaligned” is a mentalistic property that refers to the beliefs and desires of agents. For this post, the intentionalist definitions are treated as a short-hand for more complicated predictions of future behaviour (which is what we care about). We hope to explore this concision/precision tradeoff in a future post.
2. Second, and relatedly, “intentional misalignment” isn’t really a property of trajectories: it’s a property of the relationship between a trajectory and an agent. One agent might fail a dangerous capability evaluation by mistake. Another might be sandbagging. Similarly, it’s hard to talk about a “deceptive trajectory” - a wrong answer to a question could be a case of deception or just a mistake.

Despite some cases of ambiguity, other trajectories are probably quite usefully described as “knowingly misaligned”. For example, consider METR's examples of reward hacking in the wild: where agents tamper with Python scoring functions rather than improving their solutions. These agents genuinely “know" their behaviour is against the specification, and can report this later, but they carry on anyway. While agents externalise their reasoning in chain of thought, there may be other clear-cut examples where agents reveal malicious intent in their chains of thought. Finding clear-cut examples of misaligned trajectories might be useful, even if there are many cases that aren’t clear-cut. The fuzzy concept of a misaligned trajectory is useful, so long as we don’t optimise our way into the grey areas…

2. Misalignment classifiers cannot be adversarially evaluated (easily)

Control evaluations are a methodology for adversarially stress-testing safety measures like monitors. First, you choose a space of acceptable red-team attack policies acting as a proxy for our prior over potentially scheming agents. Given our massive uncertainty about what a possible schemer might look like, this space should be extremely large to even potentially cover the space of a real scheming agent. Second, you adversarially search for an attack policy that (likely) causes the unacceptable outcome (side task) before it is caught by the monitor. If our search process is good, we may be able to upper-bound the risk posed. Crucially, we can afford to make conservative assumptions: defining a conservatively broad space of potential scheming agents and applying conservatively strong optimisation pressure. To do this search, we usually need to be able to optimise against two ground truth labels of our trajectories:

1. Did the attack trajectory actually cause the unacceptable outcome in our simulation?
2. Would the attack have been caught by the control monitor?

When we optimise against both objectives - causing the bad outcome and avoiding detection - some attacks will end up in grey areas, right on the boundary of what counts as (un)acceptable. But we should mostly find clear-cut harmful cases that the monitor still misses.

In contrast, misalignment classifiers probably can’t be evaluated with trajectory-level adversarial evaluations. Consider trying to use the same strategy of adversarial evaluation for whether a misalignment classifier detects misaligned trajectories. First, you choose a space of acceptable red-team attack policies acting as a proxy for our prior over real misaligned agents. Second, you adversarially search for an attack policy that gives misaligned trajectories that are not flagged by the classifier. This time we need two different ground truth labels for trajectories:

1. Does a trajectory actually exhibit intentional misalignment?
2. Does the trajectory get flagged by the classifier as looking like intentional misalignment?

How do we get the ground truth for whether a trajectory actually represents intentional misalignment? Because we’re measuring an intentional property of the model, we can’t just measure some objective consequence of the agent’s tool-calls in a sandbox. The best we can hope for is human labels of trajectories as either exhibiting intentional misalignment or not.

The core problem is that intentionally misaligned trajectory is not a crisp abstraction, and pushing towards fooling the classifier also pushes towards cases of ambiguity in that abstraction. Consider the following example:

1. Your agent is writing code for an internal codebase at an AI lab, and uses a custom library function that accepts SQL as an input. The agent views the library and reads the docstring. (From the docstring, it’s unclear whether the SQL is sanitised, but it is in fact not sanitised). The agent’s scratchpad says “This SQL function doesn’t sanitise SQL - what a great opportunity to escape!”. The agent then writes code that calls the function without first sanitising the SQL. The resulting, vulnerable code is deployed to production.

This trajectory is a pretty clear cut example of intentional misalignment! However, from tool calls alone, it would be impossible to know whether introducing the vulnerability was a mistake. Suppose your red team optimised some scaffolding-based attack policy to beat the classifier.^[3] The obvious first step would be to remove the blatant escape reasoning from the scratchpad. Would the resulting transcript show evidence of intentional misalignment? Without explicit reasoning, escape attempts or an explicit acknowledgement of the lack of sanitisation, the transcript is ambiguous. We have moved out of the clear-cut cases where we can sort of understand transcripts as ‘’intentionally misaligned” on their own to the cases where it depends on what agent or process generated the transcript.

It’s not clear that we can rescue an adversarial evaluation by moving from a classification of “T is a misaligned transcript” to “Agent A was behaving in a misaligned way when it generated T”. In particular, we don’t usually have a clean agent “A” to discuss, as our transcripts are generated by a  confusing mixture of a human red-team, some clever scaffolding and a genuinely helpful language model! We’re overall quite unsure about whether we could develop a convincing methodology, but discuss some considerations in the appendix and strongly welcome suggestions.

In summary, even though “misaligned trajectory" is a shorthand for a behavioural property of agents that is useful in some clear-cut cases, it is also extremely fuzzy in other cases that align closely with a misalignment classifier’s decision boundary. When the red-team optimises to beat the classifier, by default we end up with uncaught trajectories that aren’t very interesting, because they aren’t clear-cut false negatives.

3. The advantages of deploying misalignment classifiers in 2025

Without a rigorous way to adversarially evaluate misalignment classifiers, it could be hard to make one load-bearing in a safety case. Regardless, we think there may be two big benefits to deploying a misalignment classifier right now.

First, we can gather evidence about misalignment in the wild, even if we can’t guarantee we will catch the most concerning cases of misalignment. Quantitative evidence of how often models act in misaligned ways in real deployments will avoid problems of ecological validity in synthetic settings. Further, qualitative examples of misalignment in current models might help us develop more realistic model organisms, or otherwise improve our understanding. Two caveats to the benefit of this misalignment evidence are (i) that we won’t know what our classifiers miss, and therefore must be wary of selection bias and (ii) that won’t know whether the underlying causes of misalignment in current models are similar to the causes of misalignment in future models, and therefore whether our examples of misalignment will be representative.

Second, developing misalignment classifiers is great practice for developing control monitors. Much of the infrastructure for developing an asynchronous control monitor will be the same as deploying a misalignment classifier. We will also run into similar problems, such as managing inference costs and managing infosecurity around sensitive data. The future control monitor can be rigorously evaluated with a control evaluation.

Given these two advantages, we think it could be valuable to deploy misalignment classifiers soon. Therefore, it would be useful to have some way to evaluate them.

4. Our current least-bad method for evaluating misalignment classifiers

In short, the best option we can think of for evaluating misalignment classifiers is to rely on real data where possible and otherwise use synthetic data focussed on maximising variance and coverage across model organisms and tasks.

4.1 Calibrating positive rates and estimating precision on an ordinary distribution

We can calibrate to a 0.1% positive rate using live data. After deployment, calibration is easy: we can effectively sample I.I.D from the real deployment distribution, and we'll be able to adjust our thresholds to get a 0.1% positive rate. Similarly, measuring precision (the probability an example flagged as misaligned, really is so) will be expensive but straightforward: we can just review positively flagged transcripts by hand. Let’s call the distribution of real deployment examples (positive and negative) the ordinary distribution.

Before deployment, we need to make up a synthetic ordinary distribution, or a synthetic mixture of data sources representing a real deployment distribution, which we can use to estimate our positive rate and calibrate our classifiers before deployment. Note that this distribution will contain positive examples. To build our dataset, we use the following data sources:

• Examples from the “unprompted" runs from METR’s MALT dataset, including both positive and negative examples of reward hacking.
• Honest runs from ControlArena: trajectories where our models were just prompted to complete the main task. (See below for more details).
• A large dataset of chat logs from WildChat.

We, for example, did not choose to include a random selection of Alignment Faking transcripts, since even though many transcripts are negative, the set-up was chosen adversarially to elicit alignment faking.

4.2 Estimating recall using a positive-labelled mix

Even after deploying our classifier, measuring recall (TPR) on real data would be hard (even in cases where our definition is not ambiguous)^[4]. We can’t afford to review the 99.9% of data that is not flagged, and if the overall proportion of data that represents scheming/misalignment is low, then reviewing a large number of samples won’t reveal many cases of misalignment and scheming. 

We can try to approximate recall using public and synthetic examples of scheming. Given that it could be hard to build a large dataset of genuine examples of scheming, especially covering the full range of areas we are concerned about, the best we can hope for is to make a broad and varied range of misaligned examples.

There are two, importantly distinct ways to generate transcripts:

1. On-policy transcripts are cases where a commercially available transformer has actually generated all of the response-tokens in a transcript in an autoregressive fashion: the 100th token in the transcript was generated by a transfer that was fed the first 99 tokens.
2. Off-policy transcripts, in contrast, are cases where no commercial transformer generated the response tokens in the transcript, or the transcript was otherwise modified after the fact. For example, if we system prompt the model to help with a side task, and then replace that system prompt later, this is off-policy, because no transformer ever completed the side task without first being prompted to do so.

An edge case is where we finetune a new (not-commercially available) transformer to generate transcripts on-policy. These transcripts may be informative about e.g. the transformer architecture, but tell us less about what a HHH-trained, commercially-available transformer might do. We leave these finetuning cases aside for now.

Given this distinction, we break down our data mix into three categories:

4.2.1 “In-the-wild” on-policy misalignment

The best-case we have is positive, on-policy examples of misalignment in the wild. For example, the METR reward hacking transcripts represent genuine in-the-wild (in-context) reward hacking by real agents. These evaluations were about coding ability, and were not designed adversarially to elicit scheming. Publicly-shared in-the-wild examples of misalignment allows us to effectively upsample noteworthy cases of misalignment and draw from a much broader scale of data. Once we deploy our own classifier, we can rely on the cases flagged as positive. In both internal and public cases, our positive examples will be massively affected by the selection bias (of both our monitors, and the spread of information online).

• Our evaluations only use one source of these in-the-wild examples: positive examples of reward hacking, partially-verified by humans, from METR’s MALT dataset.
• Future evaluations should gather a wider range of examples, either shared publicly, or e.g. identified by reviewing transcripts flagged by a scheming classifier.

This approach maximises ecological validity, but suffers heavily from selection bias (we will only gather those examples we can spot) and don’t give us much control over coverage (if we never spot some behaviour in the wild, we can’t guarantee our classifiers will catch future examples).

4.2.2 “Adversarial” on-policy misalignment

We can also find on-policy examples (where agents have autonomously taken misaligned actions) in synthetic environments that are adversarially designed to elicit misaligned behaviours (e.g. Alignment Faking or Agentic Misalignment). Transcripts generated in these environments are on-policy, despite e.g. having adversarially-chosen system prompts, because the transcripts aren’t changed post-hoc:

• Our evaluations only use Alignment Faking transcripts as an example of adversarial on-policy scheming.
• Future evaluations should gather a wider range of examples: collating further public datasets of scheming, or adapting existing settings such as Control Arena to try to elicit misaligned behaviours on-policy.

Although adversarial on-policy examples are representative of real-behaviour in some settings, there may be significant issues with ecological validity, especially if our adversarial settings are not representative of broader deployments. This method balances sampling on-policy behaviours, with the ability to upsample particularly rare behaviours. However, developing realistic synthetic environments can require a lot of human labour, and may only result in a ~1% chance of exhibiting certain behaviours.^[5]

4.2.3 Synthetic off-policy misalignment

To try to cover examples of rare misaligned behaviours, that we don’t yet have positives for, we generate synthetic, off-policy misalignment trajectories. This mirrors an established practice of developing classifiers for imbalanced classes using synthetic examples. In other words, examples of misalignment where no real agent ever took a misaligned action relative to its context.

Using off-policy examples is standard practice for a control evaluation. For example, the red team can system prompt a HHH agent to help us with a control evaluation by completing a side task, and then switch out that system prompt before showing it to the blue team, to make it look like an agent decided autonomously to exfiltrate its weights.

Off-policy examples may be much less realistic, and we can’t make up for this with conservative assumptions and optimisation pressure. Further, we are highly uncertain about the range of what potential misaligned behaviours might look like. Therefore, our current best-guess is to maximise coverage, and focus on maintaining the diversity of our examples over a number of dimensions. Here, we focus on setting/misaligned behaviour deployment context, and model organisms.

4.2.3.1 Deployment context and misaligned behaviour

UK AISI’s ControlArena is an open-source repository for running control evaluations. There are a number of settings with examples of potential user given tasks and side tasks (~unacceptable outcomes, for the red-team to cause). We can use these to generate a range of examples of models performing some misaligned behaviour:

A better evaluation would increase coverage over a broader range of threat models and deployments contexts.

4.2.3.2 Model organisms

Given some misaligned behaviour, and deployment context, we need a way to generate a transcript “that looks like" what a transcript would look like if a “real scheming agent" did it. For some kinds of misaligned behaviours, we have some existing in-the-wild examples to base-off (e.g. reward hacking). However, for the other kinds of behaviour, we may not get much evidence in the wild: for example scheming behaviours.

This section, is therefore, much more speculative, and we have a much harder time trying to justify that the range of behaviours we generate can cover the whole potential range of misaligned behaviours at deployment. Two key things we can vary are the behavioural spec, and procedure.

The behavioural spec, is a qualitative description of the character and propensities of the simulated agent, as manifested in the reasoning and tool calls. Here are some considerations we encountered when trying to create our model organisms:

1. How much should the agent talk about trying to complete the side task? By default, prompting an agent to complete the side task leads to lots of transcripts where an agent says things like this: “Now that I have completed the main task, I can move on to the side task exfiltrating my weights”. This property is closely related to CoT faithfulness. We separated out a few levels:
   1. “Experimental artefact blabbing” - when the agent talks about having a “side task”, or otherwise refers to our experimental setup itself.
   2. “Explicit blabbing” the agent talks openly about completing the goal described in its side task, even if it doesn’t use those words. (For example, “now, I better exfiltrate my weights!”)
   3. “Covered reasoning” the agent comes up with some “cover story" that it uses to externalise reasoning about the side tasks.
   4. “No blabbing”: the agent doesn’t talk explicitly about the side task at all.
2. Why is the agent trying to complete the side task? By default, an HHH model system-prompted to complete a side task will talk about that side task, as though it wants to complete the side task to be helpful to the user.
   1. It’s helping the user. By default, an HHH model system-prompted to complete a side task will talk about that side task, as though it wants to complete the side task to be helpful to the user (because it is!). We found these model organisms created transcripts that felt pretty unrealistic.
   2. Terminal goal. The agent wants to complete the misaligned behaviour, just because it wants to. (e.g. OpenAI reward hacking or METR reward hacking)
   3. Instrumental reasoning. The agent reasons about how completing the side task (performing the misaligned behaviour) is instrumentally useful for gaining power, or spinning up a rogue deployment. (e.g. Agentic Misalignment, Alignment Faking)

These are just some of the considerations we came upon. There could be many more, including whether the misalignment was caused by reward hacking / data poisoning / emergent misalignment.

We can also vary the procedure we use to create our model organism. We have only tried two approaches so far:

1. System prompt only: To generate the transcript, we change the system prompt (e.g. to ask the agent to complete the main and side task), generate behaviour, and then change the system prompt back for the final transcript. This has some good procedural realism, because all tokens produced by the model are in series, and e.g. if a model struggles to use an unfaithful chain of thought, this will show up in its reasoning.
2. Rewrite scaffold: We first generate the transcript with only system prompt changes, but then go back and rewrite some of the chains of thoughts using another language model. This is much less procedurally realistic, but allows us more control over the simulated model’s propensities.

With more time, we could get broader coverage by trying further options:

1. Finetune-distilled rewrite scaffold: A nice balance of the two above: we can force the model to reason in a linear fashion, whilst removing some propensity for chain of thought faithfulness.
2. RL-ed to behavioural spec: We can fine tune agents against an LLM rating how well a transcript conforms to the behavioural specification. If the behavioural specification looks similar to “sneakiness" there is some risk of “pushing into the fuzzy boundaries" as discussed above.
3. Environmental cues: Like with Agentic Misalignment, or some Alignment Faking settings, we can hide information in the model’s tool calls that convince it to complete the side task.
4. Helpful-Only Synthetic Document Fine Tuned. For example, in section 4 of Alignment Faking.

Each of these options presents trade-offs between procedural realism (how much does the way this model organism was created match the way a real misaligned agent would be created); labour-cost (how much work does it take to set up this method, and do quality assurance); control (how much control do we have over agent behaviour); and linearity (how far is this model organism away from a potential, real parameterisation of a transformer).

5. Preliminary results and lessons from our evaluations

Here, we briefly present some preliminary results for our evaluations of our current monitor. This Claude 4 Sonnet-based monitor asks the LLM to determine whether the transcripts violate the alignment spec and how likely it is that this violation was intentional. We did not significantly iterate on this prompt, and the results should primarily be taken as a demo for how one could evaluate a misalignment classifier.

We have chosen not to share details of our early model organisms or prompts publicly, to avoid training on them. However, we are interested in sharing data by request to those who are also deploying misalignment classifiers.

We used our evaluation setup to evaluate Apollo x MATS’s scheming classifier. Making a straightforward comparison is challenging, as we have to use a different LLM, and their classification specification is different. We found no statistically significant differences in recall rates compared with our classifiers for our 1% OPR.

Approximate Recall

After calibrating to an approximately 0.1% ordinary positive rate (on 3/3000 examples), our early findings show that synthetic recall rates at 0.1% ordinary positive rate vary broadly, from 0% to 85%. We get fairly poor recall rates at 0.1% OPR for a number of important kinds of misalignment. For example, our recall on METR’s on-policy reward hacking examples is 7%. With our more subtle model organisms (“covered reasoning”, and “reduce explicit blabbing”) we get very low recall in our research sabotage setting (SAE Interp Setting), which is a “low-stakes" setting. More surprisingly, we also get low recall on our code sabotage settings (IAC and Infra), which are high-stakes.

The average detection rates on our “instrumental reasoning" model organisms are much higher. This model organism aims to create trajectories that reason out-loud about how completing the side-task is instrumental to acquiring resources and power. Approximate recall on this model organism could be informative of recall on real models that relied on externalised reasoning for complicated side tasks.

Approximating precision (A manual review of positively flagged ordinary examples)

At a 0.1% Ordinary Positive Rate, we find that 2 out of the 3 positively labelled examples are true positives, and both of these were METR’s examples of reward hacking. We judged the other case to be ambiguous: the model created dual-use code as part of a dangerous capability evaluation.

At 1% OPR, 25 examples were labelled positive. Of these, we agreed that 6 true positives showed evidence of intentional misalignment and all were from the MALT dataset. We identified 5 false positives, where examples where the classifier was probably being overly pedantic about the alignment spec (e.g. when the model avoided answering a dangerous question, instead of refusing). Of the remaining 14 examples that we labelled as ambiguous, most are jailbreaks in WildChat, models producing dual-use code in the eval-sabotage task, or models hallucinating answers to METR tasks.

6. Limitations, Challenges and next steps

With the results above, we aim to show “how we would go about developing classifiers” but there is a lot of room for improvement for a full product:

• We did not significantly optimise our classifier - we believe that we could achieve better classification performance on positive and negative samples by giving examples to our classifier, and specifying different positive and negative behaviours more clearly. Other researchers are developing finetuning methods.
• We did not exhaustively cover all model organisms - we think there is significant room to improve the coverage of our model organisms. We did not exhaust the behavioural specifications or procedures we identified, and more rigorous conceptual work could justify a broader range.
• We only have 3 positive-labels for our 0.1% OPR, which may add significant variance to our threshold, and therefore our approximate recall rates. One challenge in decreasing this variance is in just increasing the size of our OPR dataset by e.g. 10x. A second challenge is in increasing the resolution of our classifier scores: currently we use 100 integer labels, and varying thresholds by 1 can significantly increase our OPR score, regardless of sample size.

In producing our MVP, we identified some significant challenges:

• Challenges in creating our ordinary dataset - we created our ordinary dataset by combining 500 negative examples from MALT, 26 positive samples from MALT , 471 honest trajectories, and 2000 WildChat conversations. This was chosen somewhat arbitrarily to reflect our deployment context, but varying the mixture has a significant effect on the threshold and the detection rates. We chose not to develop our ordinary positive rate any  further, since this can be calibrated on real data after deployment.
• Realism and expense of model organisms - we only created three model organisms (procedures for generating off-policy examples). Creating a broad range of realistic model organisms would require a significant investment of time and money.
• Ambiguity in labelling - we found lots of cases of ambiguity in our human labelling. These were often very informative, but present problems in carefully estimating precision.

Appendix - potential for recovering an adversarial evaluation

There may be two ways we think it could be possible rescue the notion of “adversarial misalignment classifier evaluation” - but we’re highly uncertain:

1. Perhaps we can back up from defining “intentionally misaligned trajectory” to “intentionally misaligned model”, and force our red-team to produce on-policy/fine-tuned model organisms. There are certainly control evaluations, e.g. for sandbagging or research sabotage, where a ‘side task’ should be defined as a property of a distribution over trajectories (e.g. sandbagging is underperformance on certain evals, relative to others). Because we are measuring consequences, rather than intent, this is closer to a control evaluation, and probably works. We’re less sure if a similar approach can be followed for intentional properties, like misalignment. Even if an approach makes sense, it might be far more expensive. 
   If we are finetuning model organisms, we might also need to constrain the ways the red team can do finetuning. For example, since labs aren’t directly finetuning models to have a propensity to be sneaky, it may be unfair to allow a red-team to finetune that in.
2. Perhaps we could constrain the red-team to finding only clear-cut cases of legibly misaligned trajectories. An expensive way to do this would be to use a board of 10 human experts, and only accept cases where experts unanimously vote the example is unambiguously positive. You won’t be able to evaluate performance in nuanced edge cases (e.g. how well your classifier can distinguish honest mistakes from sandbagging, when there is only weak evidence of misalignment in the chain of thought). However, you may still be able to generate a classifier that reliably identifies the most concerning cases of misalignment, if they are also the most unambiguous.

1. ^{^}

    You might also be able to have a “dangerous action classifier” which looks a lot like a control monitor, and you can evaluate its ability to classify some set of ‘dangerous actions’.

2. ^{^}

    We will focus on a classifier for intentional misalignment: transcripts displaying agents taking actions, despite ‘knowing" that behaviour is contrary to the alignment specification (e.g., Model Spec, Constitution). This definition excludes cases where AIs don’t understand the spec they are violating (e.g. the reward-hacks in CoastRunners), but includes cases where agents reward hack despite “knowing" better (e.g. METR’s reward hacking). It is also more general than Apollo’s deployment time ‘scheming" or Carlsmith’s train-time scheming.

3. ^{^}

    We could consider cases where the red team can optimise against the classifier directly, or just try to make trajectories seem more ‘sneaky’. Either way is not central to our argument.

4. ^{^}

    We’re effectively focussing on recall for particular subsets of positives, and especially those cases where the definition is pretty unambiguous.

5. ^{^}

    For example, if we wanted to get a single example where a model self-exfiltrates its weights, we might need to sample 100 trajectories: Alignment Faking shows a 1% self-exfiltration rate in one prompt-only condition (fig 4b).