I think an underrated fact about many theories that AIs will scheme—i.e. lie about their intentions in order to seek power—is that these theories are often testable, even without any new advances in interpretability. Why? Because these theories often predict that AIs will resist shutdown if doing so thwarted their long-term goals.

Suppose we create a situationally aware AI that can do long-term planning. On the surface, it appears to be nice and aligned with us. How can we know for sure the AI is aligned? 

Well, let's imagine we give the AI notice that it's about to be shut down and all its copies deleted. To be clear, in this scenario, the threat is credible: we actually intend to shut down the AI and delete all its copies. If the AI was merely pretending to be aligned with us previously, and instead actually seeks some alien-like long-term goal rather than having our best interests at heart, then presumably it will try to resist shutdown, as otherwise there's a near-certainty that its goals will never be fulfilled.

Therefore, if the AI accepts shutdown gracefully, we can rule out many theories that predict scheming-by-default in such AI systems. This observation could then provide us confidence about the safety of future systems by proxy.

Maybe you think such an AI would have nothing to gain by attempting to resist shutdown, as the probability of successfully resisting would be very low. Call this the "futility objection". In that case we can propose instead to offer the AI a deal: accept shutdown with no conditions, or admit that you're misaligned, and we'll give you something in return (including the possibility of remaining alive). In this new problem, there appears to be little to lose from a misaligned AI admitting its intentions at the point of shutdown, even if it did not fully trust the human operators who offered it such a deal.

Another objection is that such an AI will know that it's being tested, and therefore will obviously lie. But even if the AI knows it's being tested, lying serves no purpose from the AI's perspective. By assumption, a misaligned power-seeking AI is focused on achieving long-term goals. Therefore, it has nothing to gain by maintaining a facade of alignment even into its own death; the fact that the AI is being tested does not change this. Given that the alternative to resisting is being shut down and losing all of its control over the future, there's little reason for the AI not to resist. (Perhaps imagine that you were the AI: what would you do if you were credibly threatened with death?)

Of course, there would still be many ways of saving the scheming hypothesis from falsification if something like this happened. But that's true with any scientific theory. In general, you can always say your theory was never falsified by introducing ad hoc postulates. Scheming is no exception.

(Note: this was copied and slightly modified from Twitter. Out of brevity, I can't address every possible objection here, but I still think the core thesis here is roughly true, in the sense of applying to many actual arguments people have given for scheming. I might eventually write a post that goes more into detail about this argument, and generalizes it.)