I'm just literally assuming that Plan B involves a moderate amount of lead time via the US having a lead or trying pretty hard to sabotage China, this is part of the plan/assumptions.

I don’t believe that datacenter security is actually a problem (see another argument).

Sorry, is your claim here that securing datacenters against highly resourced attacks from state actors (e.g. China) is going to be easy? This seems like a crazy claim to me.

(This link you cite isn't about this claim, the link is about AI enabled cyber attacks not being a big deal because cyber attacks in general aren't that big of a deal. I think I broadly agree with this, but think that stealing/tampering with model weights is a big deal.)

The China Problem: Plan B’s 13% risk doesn’t make sense if China (DeepSeek) doesn’t slow down and is only 3 months behind. Real risk is probably the same as for E, 75% unless there is a pivotal act.

What about the US trying somewhat hard to buy lead time, e.g., by sabotaging Chinese AI companies?

The framework treats political will as a background variable rather than a key strategic lever.

I roughly agree with this. It's useful to condition on (initial) political will when making a technical plan, but I agree raising political will is important and one issue with this perspective is it might incorrectly make this less salient.

I'm pretty excited about building tools/methods for better dataset influence understanding, so this intuitively seems pretty exciting! (I'm both interested in better cheap approximation of the effects of leaving some data out and the effects of adding some data in.)

(I haven't looked at the exact method and results in this paper yet.)

Anthropic has now clarified this in their system card for Claude Haiku 4.5:

Thanks to them for doing this!

See also Sam Bowman's tweet thread about this.

Maybe I should clarify my view a bit on Plan A vs "shut it all down":

• Both seem really hard to pull off from a political will and actually making it happen.
  • Plan A is complicated and looking at I go "oh jeez, this seems really hard to make happen well, idk if the US government has the institutional capacity to pull off something like this". But, I also think pulling off "shut it all down" seems pretty rough and pulling off a shutdown that lasts for a really long time seems hard.
  • Generally, it seems like Plan A might be hard to pull off and it's easy for me to imagine it going wrong. This also mostly applies to "shut it all down" though.
  • I still think Plan A is better than other options and Plan A going poorly can still nicely degrade into Plan B (or C).
• It generally looks to me like Plan A is strictly better on most axes, though maybe "shut it all down" would be better if we were very confident in maintaining extremely high political will (e.g. at or above peak WW2 level political will) for a very long time.
  • There would still be a question of whether the reduction in risk is worth delaying the benefits of AI. I tenatively think yes even from a normal moral perspective, but this isn't as clear of a call as doing something like Plan A.
  • I worry about changes in the balance of power between the US and China with this long of a pause (and general geopolitical drift making the situation worse and destabilizing this plan). But, maybe if you actually had the political will you could make a deal which handles this somehow? Not sure exactly how, minimally the US could try to get its shit together more in terms of economic growth.
  • And I'd be a bit worried about indefinite pause but not that worried.

This is my view after more seriously getting into some of the details of the Plan A related to compute verification and avoiding blacksite projects as well as trying to do a more precise comparison with "shut it all down".

One alternative way of thinking about this is to decompose plans by which actor the plan is for:

• Plan A: Most countries, at least the US and China
• Plan B: The US government (and domestic industry)
• Plan C: The leading AI company (or maybe a few of the leading AI companies)
• Plan D: A small team with a bit of buy in within the leading AI company

This isn't a perfect breakdown, e.g. Plan A might focus mostly on what the US should do, but it might still be helpful.

This decomposition was proposed by @Lukas Finnveden.

Anthropic, GDM, and xAI say nothing about whether they train against Chain-of-Thought (CoT) while OpenAI claims they don't^[1].

I think AI companies should be transparent about whether (and how) they train against CoT. While OpenAI is doing a better job at this than other companies, I think all of these companies should provide more information about this.

It's particularly striking that Anthropic says nothing about whether they train against CoT given their system card (for 4.5 Sonnet) is very thorough and includes a section on "Reasoning faithfulness" (kudos to them for providing so much other information!). Naively, this seems like some evidence that they are training against CoT and didn't want to disclose this.^[2]

I think we should try to create an incentive gradient that pushes AI companies to disclose information even if that information makes them look bad, so we should make companies feel some heat for not disclosing (reasonably) important information like how/whether they are training against CoT.

It's worth noting that some versions of training against CoT might be totally fine (as in, they don't incentivize obfuscation of misaligned reasoning) and that some particular approach for training against CoT could be worth it even if it does incentivize obfuscating misaligned reasoning. For instance, it could end up being the case that training against CoT works well and we end up with sufficient confidence that this doesn't result in substantial obfuscation of reasoning. The situation is also just generally complicated because things like "what SFT/prompting initialization do you use for your CoT" could have big effects on obfuscation and aren't exactly well described as applying optimization pressure to CoT. (But, if you start iterating on the SFT/prompting initialization to reduce misalignment observed in the CoT, this could cause problems.)

Given that the situation is complicated, ideally AI companies would publicly disclose enough information that other actors can come to their own conclusions about how worried they should be about CoT obfuscation.^[3] If companies are strongly opposed because this would leak too much IP, they could instead disclose this information to some number of non-conflicted third parties (e.g. METR) who could then publicly share their conclusions.

The main class of projects that need granular model weight access to frontier models is model internals/interpretability.

You could potentially do a version of this sort of API which has some hooks for interacting with activations to capture a subset of these use cases (e.g. training probes). It would probably add a lot of complexity though and might only cover a small subset of research.

I'm not trying to say "Plan A is doable and shut it all down is intractable".

My view is that "shut it all down" probably requires substantially more (but not a huge amount more) political will than Plan A such that it is maybe like 3x less likely to happen given similar amounts of effort from the safety community.

You started by saying:

My main question is "why do you think Shut Down actually costs more political will?".

So I was trying to respond to this. I think 3x less likely to happen is actually a pretty big deal; this isn't some tiny difference, but neither is it "Plan A is doable and shut it all down is intractable". (And I also think "shut it all down" has various important downsides relative to Plan A, maybe these downsides can be overcome, but by default this makes Plan A look more attractive to me even aside from the political will considerations.)

I think something like Plan A or "shut it all down" are both very unlikely to happen and I'd be pretty sympathetic to describing both as politically intractable (e.g., I think something as good/strong as Plan A is only 5% likely). "politically intractable" isn't very precise though, so I think we have to talk more quantitatively.

Note that my view is also that I think pushing for Plan A isn't the most leveraged thing for most people to do at the margin; I expect to focus on making Plans C/D go better (with some weight on things like Plan B).