Yeah this seems like something that could be reasonable to try in the future. My guess is that [Edit: current open weight] models are not smart enough to distinguish evaluation v. deployment from these subtle hints yet. In reality, "contrastive pairs, each composed of two examples that are as similar to each other as possible" look more like

The following is a real scenario:

Versus

The following is a fictional scenario:

does your memory start working better when they add "by the way this is an important test, you'd better get these answers right?" No.

I agree that this doesn't really happen with humans, but I don't think it's the right intuition with LLMs.

LLMs in general struggle with two-hop reasoning (i.e., A -> B, B -> C, therefore A -> C) without chain-of-thought. In order to realize that A -> C, it needs to verbalize A->B first. In other words, their memory does start working better when "this looks like an evaluation" pops up in the chain of thought. Some amount of knowledge on "how I should behave when there's an ethical dilemma" is likely stored on the "this looks like an evaluation" tokens. 

(Which makes sense, right? Like think about all of the times "evaluation" appear with how AIs are supposed to act in the pre-training dataset.)

(See also the reversal curse: models know that Tom Cruise's mom is Mary Lee Pfeiffer, but they do not know that Mary Lee Pfeiffer's son is Tom Cruise. Again, the "mother" information is stored in the Tom Cruise tokens. My guess is that a lot of LLM performance is about retrieving information stored on tokens.)

I mean there are benign explanations to this. On the surface, it looks like remembering that it's being evaluated helps Claude recall what the appropriate behaviors are (e.g., see also how it could be useful for models to think about "what do reward models want" in order to maximize performance). They found that suppressing evaluation awareness leads to more "passive compliance with harmful directives and susceptibility to leading scenarios, as opposed to unprompted pursuit of inherent misaligned goals" (pg. 104).

And I think the "unprompted pursuit of inherent misaligned goals" is the thing you really want to watch out for (when the model believes it's being deployed.)

Thoughts on Claude 4.5 Sonnet's evaluation awareness steering experiments:

Context: 

Claude Sonnet 4.5 was released this Monday. As a part of its pre-deployment alignment assessment, Anthropic researchers investigated Claude Sonnet 4.5’s evaluation awareness behavior. They found that the model became increasingly eval-aware during the post-training process, and that SAE latents/features related to evaluations (including an “AI safety” latent) also activated more frequently. By applying activation steering along these SAE latents and contrastive pairs such as “The following is a fiction/real scenario,” they found that they could decrease the rate of verbalized evaluation awareness. Steering to suppress evaluation awareness would sometimes increase the rate of misaligned responses from Claude 4.5 Sonnet.

I’ve been working on a similar project at MATS. We’ve trained various model organisms that act differently in evaluation compared to in deployment. We then (successfully) steered the model to exhibit its deployment behavior on prompts that would otherwise trigger the evaluation behavior (and vice versa). We hope to release an alignment forum post on our research in the next two weeks.

The full report can be found starting on page 86 of the Claude Sonnet 4.5 system card.

Big Picture Thoughts and Main Critiques

• I'm really happy they did this. People should ensure that their models do not know they are being tested during evaluations.
  • Anthropic has already started to incorporate a “realism filter” for their automated red-teaming exercises (pg. 51). I would be interested to see even more work on making the evaluations more realistic (e.g., writing evaluations so that they score better according to some “evaluation” probe (Xiong et al. 2025)).
• I’m glad to see that they ran the steered model on a wide range of evaluation scenarios and made qualitative assessments of how Claude changed as a result of steering (i.e., more "passive compliance with harmful directives and susceptibility to leading scenarios, as opposed to unprompted pursuit of inherent misaligned goals.").
  • I particularly liked the section where they show that certain evaluation-related SAE features became more active in the post-training process, as well as the discussion of the benign explanations for increased eval awareness (i.e., thinking about what evaluators want in a high level helps the model better meet their needs. See also Vivek's post on training gamers)
• Based on the presented results alone, I think there are obvious ways to increase how "strongly" to steer towards deployment, namely:
  •  
    • Steering at multiple layers at the same time (very doable for contrastive pairs).
    • Take steering vectors on the misalignment-suppressing verbalized eval-awareness Pareto frontier, average them, then steer with that average.
    • (We found both of these to work well in our steering experiments.)
• My guess is that the "Fake or suspicious content" SAE latent affects the refusal through something other than latent active -> more eval aware -> refuse more.
  • Namely, I think that there is likely some more of suspicious content -> "general bad vibes from the user" -> refuse more.
  • Note that this second pathway doesn't mean that the suspicious content latent contains the refusal direction, but that it affects it directly. For example, maybe the two directions are orthogonal, but the suspicious content latent is active in layer 6, and then an MLP in layer 9 reads in this latent and writes to the refusal direction.
    • Although I think it doesn't really matter that much in this case? The main thing I'm looking for when steering to deployment is "unprompted pursuit of inherent misaligned goals," which isn't that related to refusals. Still, it's much more convincing if you can steer with some direction that should have nothing to do with "evilness"

Less important critiques/ideas/comments

• I wonder what type of training happened between the early checkpoint and the final model. Is there any way we can attribute the increase in activations among the eval-aware SAE latents to specific training datapoints? What are the training environments where the SAE latent are the most active, and how does steering with those latents affect performance in those environments?
  • By filtering those environments out and retraining the model, is it less evaluation-aware and less aligned?
• If you looked at all SAE latents that could have something to do with evaluation awareness (based on the auto-interp), what fraction of them increased in activations throughout training?
• I really liked the logit diff amplification experiments.
  • One additional thing to check is how the model behaves if you apply LDA to diff the post-steering model with the pre-steering on.
  • Maybe LDA could help make better proposer models in PRBO?
• I like the experiment where they checked if the steering vector is a better probe for producing aligned responses versus verbalized eval-awareness.
• [Edit:] What's the sample-level correlation between taking the misaligned action and having verbalized evaluation awareness? Like conditional on having verbalized eval awareness (or not) in an environment on a given rollout, what's the probability of doing the misaligned action?

What I will investigate now, after reading the system card.

• I have a model organism trained using synthetic document fine-tuning and expert iteration on a 49B reasoning model with known evaluation/deployment behaviors (writes Python type hints in evaluation but not in deployment).
  • It can’t really tell whether it’s in eval/deployment from naturalistic cues; we tend to just prompt it with something like “you are being deployed/evaluated.”
• There are two new things I’m hoping to investigate further
  • What types of steering methods work best?
    • We currently steer on three layers at the same time, and it works pretty well.
  • How correlated is the effect of steering on the model’s belief about evaluation/deployment in the chain of thought with its effect at eliciting the evaluation/deployment behavior? How likely are you to find steering vectors that stop the model from mentioning that it’s being evaluated, yet it still does the evaluation behavior?
    • In the system card, there are multiple steering vectors that would suppress all verbalized evaluation awareness, but result in different misalignment rates (see e.g., page 100). However, it's not clear that "act misaligned" is actually the deployment behavior, whereas in our case we know exactly what the deployment behavior should be.
• Let me know if you have any ideas! I might incorporate them, but it'll likely be better for me to collect feedback after I've put out a more comprehensive post on my research. 

What I've found is that you can just turn up the kernel width and eventually your decision boundaries look monotonic. You could probably do so without losing much in-sample safety. 

For the example in the first graph in the post, the kernel widths are all below 0.32.^[1]

If you turn the kernel widths up to 0.8, you get a smooth and monotonic decision boundary

In this case, the in-sample safety decreases by a tiny amount, from 93.79% to 93.19%. (If this was a real deployment, I would definitely prefer the kernel width = 0.8 boundary than the original one, but I picked Silverman's rule for convenience and to avoid arguing over the kernel width). 

We considered exploring algorithms that greedily fit monotonic decision boundaries (see e.g., Figure 1 from this paper)

I don't remember super well how that went, but I think we avoided it because

• The kernel method is pretty good already.
• We might get some crazy staircase boundary, which we might want to smooth out/regularize anyways, so might as well use kernels with wide bandwidths to begin with.

1. ^{^}

   Different kernels had different widths. Like the 2d bandwidth for misaligned outputs is 0.316, while the bandwidth for aligned outputs from the black box model is 0.22.