Now that April Fool’s is over, I’d like to explore some of the ideas behind PauseAI and E/Acc Should Switch Sides, since the arguments in it have enough grains of truth that countering them deserves more than a snarky reference to 3D chess.

Considering the Merits of an Argument Made Ironically is Fair Game

I wrote PauseAI and E/Acc Should Switch Sides as a joke, but that in itself doesn’t make it unreasonable to say: “Hey, I think this is actually true!” The idea that activism and regulation can have self-defeating consequences is not a novel take. Variants include:

A call to pause AI development right now could spend down political capital, making future calls for regulation harder to push through when they are most needed.
A significant pause is unlikely until there is a "warning shot" (a major, undeniable failure of AI that shifts public opinion) so effort spent now is wasted.
Advocating for strong AI regulations before a crisis could turn people against AI safety efforts as alarmist.
Bringing in non-experts into the advocacy space will dilute the precision of the message. Using the wrong rationale for a pause could backfire.

The common theme here is that ill-timed or poorly framed actions can undermine long-term goals by provoking negative feedback and diluting the intended message. While this might be true in the extreme, I don’t think the logic is applicable here.

The Public Response to Warning Shots is not Predetermined

How society reacts to a warning shot will depend on what narratives and policies have been prepared in advance. US politics has an especially big problem with sorting every issue into partisan buckets. For AI, this seems to be:

Liberals: AGI is hype, but AI is causing real harm to already marginalized communities because it is yet another tool of greedy corporations. What we need are extensive use-case regulations.
Conservatives: AGI will provide a significant economic and military advantage to whoever gets it first, so the US must win the race against China.

Both of these narratives lead to the world being destroyed because neither of them “feel the AGI” or contend with AI being dangerous in itself. The further these narratives entrench themselves, the more strongly new evidence will be force-fit into them.

Furthermore, the longer we wait to apply governance to AI, the more integrated it will become into society and the economy. This will entrench vested interests in continued development and generally make it more difficult to apply governance in the future. By analogy, consider the fossil fuel industry. If everyone were to decide tomorrow that burning fossil fuels was a bad idea, there’s no off switch anyone can pull to stop their use. And even if there were, it would require massively disrupting the world economy. Even compromise solutions face a tremendous uphill battle against energy company lobbying. If we had taken global warming seriously early on, it would have been much cheaper to progress down different branches of the tech tree (even if that meant using fossil fuels as a temporary stopgap).

Raising awareness of AI risks before a crisis occurs helps shape the types of solutions that will be considered when action is finally taken. Crafting effective policy responses to AI risks is also difficult, so delaying this work until a crisis strikes may lead to poor decision-making. There may not be much time to react once things start escalating, so proactive advocacy is essential. In figurative terms, one of our goals at PauseAI is to carve out the riverbed in preparation for the coming flood.

Alignment is Downstream of Policy is Downstream of Mass Mobilization

When ChatGPT came out, timelines got shorter, and I started taking AI risk more seriously, I started off trying to work on alignment, leveraging my background in game development to work on projects like a Gridworlds level editor, a probability calculator for estimating p(doom), and interviewing alignment researchers. In the process, I came across some really cool ideas that were at least trying to address the core problems. But I also noticed something: all the best ideas are being pursued by volunteers while the big alignment bucks are directed towards getting AIs to align themselves—and even that only lasts until the safety teams get disbanded. These are not the economic incentives that produce safe AI.

Government policy has the power to set incentives…but politicians are following intense selection pressures of their own. That’s where mass mobilization comes in. The more the public understands—and publicly expresses their concern for—the risks of AI, the more politicians are incentivized to share those concerns. Or from a less cynical perspective, politicians are surprisingly receptive to our message; what they need is to know their constituents will have their back if they lead the charge for meaningful AI governance.

Activism. It Works, Badgers!

Holly Elmore’s The Case for AI Safety Advocacy to the Public makes the overall case for advocacy better than I can, but here’s a few highlights:

The key thing to understand about actions like protests is that they are part of a larger ecosystem, a single node in a system of feedback loops. For example, community building by itself becomes a social club; actions by themselves feel like work; putting the two together, when done well, is fun and focused. And action workshops should have a sense of progression, of rising energy that is building towards something real. At a certain point, it just feels weird not to protest. This creates a shared experience among participants, while also drawing in new interest, to start the cycle again, this time with greater experience and ideas about what to do differently next time. If the protest is big, it can act as a source of pressure. But even if it is small, at least you are showing that x-risk is a thing that can be talked about in public.

As a specific form of advocacy, contacting your legislator is low-hanging fruit. It is fast, easy, and has an outsized impact because representatives want to hear from their constituents and very few people use this lever of influence. Especially for emerging issues where a politician doesn’t have an established opinion, even a small number of personalized letters can significantly change their stance. Based on personal interviews, just 5-10 could be enough!

Think about that. If you, personally, right now, organized a local meetup for an informational session on AI plus letter writing workshop, then followed up by asking your local representative for a meeting, you could meaningfully move the needle on AI safety.

Pausing is About Process More Than Outcomes

“OK, suppose we got, say, a 6-month pause. What does that get us?”

I’ll give you one better: let’s assume a 5 minute pause—but that is international in scope and actually enforced. Useless, right? Not so fast. Remember that international agreements don’t just fall from the sky. If we got a 5 minute pause, that would imply things about the state of the world:

An international agreement, which requires
Provisions for monitoring and enforcement,
Negotiated between nations,
Each of whom genuinely buys in to the underlying need enough to be willing to sacrifice some amount of national sovereignty as part of the monitoring and enforcement clauses
And is politically capable of acting on that interest because it represents the will of their constituents
Because the general public understands AI and its implications enough to care about it
And feels empowered to express that concern through an accessible democratic process
And is correct in this sense of empowerment because their interests are not overridden by Big Tech lobbying
Or distracted into incoherence by internal divisions and polarization

If all of that was in place, we would be in a much better position to make good decisions. The pause itself is less important than creating the world in which a pause is possible. Directly trying to make the pause happen is the best way to get clear feedback regarding the obstacles and other things that need to happen first.

You Might Not be the Target Audience

Just because the average person disapproves of a protest tactic doesn't mean that the tactic doesn’t work. Roger Hallam's "Designing the Revolution" series outlines the thought process underlying disruptive actions like the infamous soup-throwing protests. Reasonable people may disagree (I disagree with quite a few things he says), but if you don't know the arguments, any objection is going to miss the point. To be clear, PauseAI does not endorse or engage in disruptive civil disobedience, but I discuss it here to illustrate some broader points. Anyways, “Designing the Revolution” is very long, so here's a tl/dr:

If the public response is: "I'm all for the cause those protestors are advocating, but I can't stand their methods"...notice that the first half of this statement was approval of the only thing that matters—approval of the cause itself.
The fact that only a small minority of the audience approves of the action is in itself a good thing, because this efficiently filters for people who are inclined to join the activist movement—especially on the hard-core "front lines”—whereas passive "supporters" can be more trouble than they're worth. These high-value supporters don't need to be convinced that the cause is right; they need to be convinced that the organization is the "real deal" and can actually get things done. In short, it's niche marketing.
The disruptive protest model assumes that the democratic system is insufficient, ineffective, or corrupted, such that simply convincing the (passive) center majority is not likely to translate into meaningful policy change. The model instead relies on putting the powers-that-be into a bind where they have to either ignore you (in which case you keep growing with impunity) or over-react (in which case you leverage public sympathy to grow faster). Again, it isn't important how sympathetic the protestors are, only that the reaction against them is comparatively worse—from the perspective of the niche audience that matters.
The ultimate purpose of this recursive growth model is to create a power bloc that forces changes that wouldn't otherwise occur on any reasonable timeline through ordinary democratic means.
Hallam presents incremental and disruptive advocacy as in opposition. This is where I most strongly disagree with his thesis. Moderates get results, but operate within the boundaries defined by extremists, so we need to learn how to work together.

If a protest action seems off-putting, that may simply be because it isn’t for you.

Once again, PauseAI does not endorse or engage in these sorts of actions; we make a point of keeping our protests within the bounds of the law. The reason for this is a trade-off. We want to be a movement that is accessible to everyone and some people are (reasonably!) squeamish about joining a movement that encourages people to break the law. You can think of our politics as representing what moderate centrism would look like in a sane society.

Building a Movement Requires Skill, Which Takes Time to Learn

Organizing large-scale movements is difficult and requires significant experience. Ever tried to run a protest? Or even a meetup? Getting a group of strangers to show up and not get totally out of line is hard. As is persuasion. As is not becoming totally homogenous in a self-reinforcing pattern that limits growth. As is continuously smoothing over potential ruptures among ideologues with bruised egos. As is wrangling volunteers who often seem to just want to debate and socialize (which is actually not so bad since it at least builds community) or go off and do their own thing…which always seems to be arguing with people on social media.

We don’t have our act together, but we’re working on it. And making progress. The best time to have gotten started was ten years ago; the second best time is now.

Don’t Underestimate the Public

Regular people are way easier to talk to about AI—even x-risk—than people immersed in the tech. From my experience in handing out flyers, which matches the reports of basically everyone who has done this, I’ve found:

No one likes flyers, but they get a lot more interested if you can catch their attention enough to say it's about AI.
Everyone hates AI.
Your biggest initial skepticism will be from people who think you are in favor of AI.
Your biggest actual pushback will be from people who think that social change is impossible.
Roughly 1/4 to 1/2 of people I have spoken to are amenable to (or have already heard about!) x-risk. Most of the rest won't actively disagree, but you can tell that particular message is not really "landing" and will pay a lot more attention if you talk about something else (unemployment, military applications, deepfakes, etc.)
The arguments people make correspond surprisingly well to arguments made by experts, the former just don’t use the proper jargon consistently, which is fine.

If you want to get an informed opinion on how the general public perceives PauseAI, get a t-shirt and hand out some flyers in a high foot-traffic public space. If you want to be formal about it, gather data and share your results. It might not be publishable on an academic forum, but you could do it next week.

Don’t Subvert Democracy

I’m not a fan of The Failed Strategy of Artificial Intelligence Doomers, but one thing it gets right is the history of early AI safety advocates inadvertently encouraging an early start to the AI arms race by gathering and convincing power-hungry people with money that ASI could be tremendously powerful. But as with any historical analysis, it’s important to draw the right lesson. I’d frame the story as follows:

Early AI safety advocates believed that humanity was on a path to self-termination from AI (or if not, then by something else), but also believed that coordination was impossible. The necessary solution was to make an end-run around coordination with a pivotal act. Some heroic individual would need to break the chains of history by seizing power with the ultimate technology. As long as this person had (1) at least a modicum of human decency and (2) properly understood the problem, then they could be trusted to usher in the Glorious Transhumanist Future. Of these two conditions, properly understanding the problem was the trickier one, so Yudkowsky and others got to work finding the potential elites of the AI space and making sure they got the right message…

The mistake here was trying to shortcut around coordination. Yudkowsky has largely come around, but is still making this mistake in subtle ways, such as by (1) framing the "shut it all down" plea as a means to buying time rather than as a new equilibrium, and (2) advocating using that "bought time" for human intelligence augmentation via genetic engineering.

As long as anti-social power-seeking is a dominant strategy, any technical solution will be in the image of and in service to those forces. Superbabies, whatever their ethical merits, will not save us. Human intelligence is what got us into the mess we are in. It is not the solution; it is the battlefield on which the war against Moloch is waged.

Virtue Ethics Thrives in High Uncertainty

Societal responses to AI are highly complex and chaotic, meaning unintended consequences will likely dominate any intervention. In such a context, the best strategy is not to attempt precise control but to make broadly good decisions and to be ready to act decisively when clear opportunities arise.

If deep analysis is your personal comparative advantage and activism is too Gryffindor for your taste, fine. But don’t throw shade on other people with different strengths doing the necessary work you’d rather avoid. There are many paths to contributing to a better future and hard things require collaboration.

Be Empirical

Not convinced? Got objections I didn’t address? You don’t need to speculate regarding what PauseAI is about—the door’s open, come talk to us! When you join PauseAI and get on the Discord server, someone from our onboarding team will (try to) contact you. If you like, you can have a one-on-one video call. I have openings and am happy to discuss anything about the organization with you. Or you can introduce yourself on the Introductions channel. Or you can argue on the Disagreements channel. Or you can attend a new member welcome meeting. Or if you don’t want to join PauseAI, you are also welcome to comment on this post or send me a direct message here on LessWrong.