AI ALIGNMENT FORUM
AF

1355
rosehadshar
Ω84340
Message
Dialogue
Subscribe

Posts

Sorted by New

Wikitag Contributions

Comments

Sorted by
Newest
No wikitag contributions to display.
AI-enabled coups: a small group could use AI to seize power
rosehadshar2mo*10

I think I agree that, once an AI-enabled coup has happened, the expected remaining AI takeover risk would be much lower. This is partly because it ends the race within the country where the takeover happened (though it wouldn't necessarily end the international race), but also partly just because of the evidential update: apparently AI is now capable of taking over countries, and apparently someone could instruct the AIs to do that, and the AIs handed the power right back to that person! Seems like alignment is working.

I don't currently agree that the remaining AI takeover risk would be much lower:

  • The international race seems like a big deal. Ending the domestic race is good, but I'd still expect reckless competition I think. Maybe you're imagining that a large chunk of powergrabs are motivated by stopping the race? I'm a bit sceptical.
  • I don't think the evidential update is that strong. If misaligned AI found it convenient to take over the US using humans, why should we expect them to immediately cease to find humans useful at that point? They might keep using humans as they accumulate more power, up until some later point.
  • There's another evidential update which I think is much stronger, which is that the world has completely dropped the ball on an important thing almost no one wants (powergrabs), where there are tractable things they could have done, and some of those things would directly reduce AI takeover risk (infosec, alignment audits etc). In a world where coups over the US are possible, I expect we've failed to do basic alignment stuff too.

Curious what you think.

Reply
51The Industrial Explosion
4mo
0
52AI-enabled coups: a small group could use AI to seize power
6mo
9
16Three Types of Intelligence Explosion
7mo
0
49The self-unalignment problem
3y
8
31Why Simulator AIs want to be Active Inference AIs
3y
4
22Lessons from Convergent Evolution for AI Alignment
3y
5
21The space of systems and the space of maps
3y
0
36Cyborg Periods: There will be multiple AI transitions
3y
2
15The economy as an analogy for advanced AI systems
3y
0