Summary

A defensive military coalition is a key frame for thinking about our international agreement aimed at forestalling the development of superintelligence. We introduce historical examples of former rivals or enemies forming defensive coalitions in response to an urgent mutual threat, and detail key aspects of our proposal which are analogous. We also contrast our proposal with the oft-employed analogy to nonproliferation agreements, and highlight shortcomings.

Introduction

We at the Machine Intelligence Research Institute’s Technical Governance Team have proposed an illustrative international agreement (blog post) to halt the advancement of AI capabilities, in order to forestall the development of superintelligence until it can be done safely. For those who haven’t read it already, it is enough to know that we believe substantially delaying the arrival of superintelligence is warranted, as the world is not remotely prepared to manage this transition. A halt could give the world the time it needs, and this will require international coordination, likely led by the U.S. and China.

In thinking about our proposal (and ones like it), it is natural to seek for analogies and precedents. We ourselves have spent a lot of time comparing parts of the plan to precedent and have documented these comparisons in our paper. Useful precedents include nuclear nonproliferation efforts, strategic arms reduction, and technology bans.

As precedents, these examples are valuable because they can demonstrate that the plan can be effectuated; that we have a history of taking similar actions in the past. They can also give us some evidence that parts of the plan can be effective, based on examples of success.

However, as analogies for understanding the big picture of what our agreement represents, none of the most salient analogies are satisfactory. Each produces an incomplete model of understanding why the plan would be implemented and for how to appraise the political durability of the plan over the duration it would need to endure, which could be decades.

In recognition of this, I will explore the analogy between our proposal and defensive coalitions in which parties join together to confront a mutual threat. In particular, I will examine historical examples of defensive military coalitions formed between rivals or even between adversaries that were currently engaged in armed conflict. I will highlight how considering the dynamics at play in the formation and conduct of these defensive military coalitions is key to understanding our agreement most fully.

Defensive Coalition

“If Hitler invaded Hell, I would make at least a favorable reference to the devil in the House of Commons.”
– Winston Churchill, June, 1941

A coalition is a collaboration between separate parties which share a common goal. For the purpose of this essay, let a defensive military coalition be one that is formed in order to defeat an urgent external threat. Here are a few relevant examples of defensive military coalitions and the internal rivalries overcome in their formation:

The Hellenic League (481-479 BC), in which Athens, Sparta, Corinth and other Greek city-states fought against the Achaemenid Empire (Persia)
- Athens and Sparta were fierce rivals for hegemony in Greece, and several participating city-states were at war with each other. They suspended hostilities to repel Xerxes' invasion.
The Holy League (1571-1573), in which the Spanish Empire, the Republic of Venice, and the Papal States fought the Ottoman Empire
- Venice and Spain were intense rivals for Mediterranean dominance. Venice often traded with the Ottomans and feared Spanish power, but united to win the Battle of Lepanto.
Crimean War alliance (1854-1856), in which the Ottoman Empire, France, UK, and Sardinia fought Russia
- Britain and France had been archenemies in the Napoleonic Wars (ending 1815). 39 years later, they were fighting side-by-side against Russia.
The Allies Powers (WW1, 1914-1918), in which France, UK, Russia, Italy, US, Japan, and others fought the Central Powers
- Russia and Japan fought on the same side though they had fought the Russo-Japanese War 9 years earlier.
Allies (World War II, 1939-1945), in which the UK, Soviet Union, US, and 40+ other countries fought the Axis
- Ideological differences (e.g. Western Capitalism vs. Communism) were set aside to fight the Axis.
Second United Front (1936-1946), in which the KMT (Nationalists) and CPC (Communists) fought Japan
- The KMT and CPC were in a decade-long civil war which they suspended in order to fight against Japan.
Global Coalition to Defeat ISIS (2014–2019), in which the US, Iraq, SDF, UK, France, and 80+ partners (plus parallel campaigns by Iran and Russia) fought ISIS (The Islamic State)
- US and Iran fought on the same side in Iraq (e.g., Battle of Mosul). Turkey and Kurdish forces (YPG) fought the same enemy despite Turkey considering YPG terrorists.

Motivated by Serious Threats

We are often asked why we think that states would be willing to participate in our agreement, when doing so is both costly and requires working with one’s rivals. The answer is clear when you consider the threat of artificial superintelligence (ASI) to be as serious, or worse than, the threat posed by the Nazis. Faced with a serious threat, ideological rivalries and historical grievances become a luxury that can no longer be afforded, as shown by most of the examples above.

Coalitions such as the one between the Western Allies and the USSR to fight the Axis in World War 2 exemplify how an imminent, greater threat can bring former adversaries together in cooperation. Churchill, a lifelong, fervent anti-communist, understood this instantly upon learning of Germany's invasion of the Soviet Union. His previous condemnations of the Soviet regime did not vanish, but they were rendered momentarily irrelevant by a greater, more immediate danger. As he told his nation, "The past, with its crimes, its follies, its tragedies, flashes away … The Russian danger is therefore our danger."

Of particular note is the formation of the Second United Front during the Sino-Japanese War, a brief and uneasy alliance between the Chinese Communist Party and the Kuomintang to resist the invading Imperial Japanese forces, demonstrating how even warring factions can temporarily coalesce around a powerful, immediate threat. This historical example underscores the potential, however improbable, for seemingly irreconcilable groups to forge a tactical alliance when faced with external pressure.

The United States and China, the world's leading powers (and leading AI powers), have serious and long-standing reasons for strategic competition and deep-seated mistrust. Yet, in the face of existential threats, of an artificial intelligence that could unilaterally seize control of humanity's future and destroy them both, they ought to find common cause.

Material Coordination to Neutralize the Threat

A coalition bolsters confidence in success, which may be necessary for individual members to mobilize and deploy resources to directly combat a threat. In contrast to mutual defense agreements (e.g. NATO), coalitions are not about deterring threats, but about defeating them.

This is a critical point to make about the threat of loss of control to superintelligence. But doing so requires first that we establish some expectations for the worldviews of the key parties at the time of the realization of our plan. Let me present one world I find plausible: one of the US and China are fairly confident that “If Anyone Builds It, Everyone Dies”, and the other has enough doubt to give it pause. Other states and powerful actors (e.g. major tech companies) hold a variety of positions on forestalling ASI, ranging from support, to concern, and to outright hostility. Among those who are hostile, there is a subset who are also lured by ASI’s potential to grant unlimited wealth and power, whatever the risks may be.

As a result, in worlds like this one, the creation of superintelligence cannot be addressed through deterrence, because there are enough powerful actors who are sufficiently captured by the false promise of winning the race that they cannot be deterred^[1]. Rather, societies which wish to survive must actively confront these efforts and defeat them: they must make it impossible for anyone to create ASI with unsafe methods.

This will take real effort as we describe in our agreement. The example of and analogy to military coalitions brings to mind some of the most expensive things that we have ever done. For example, World War 2, at its peak, cost the US more than 40% of annual GDP. Great expense is justifiable in the face of an imminent existential threat. While we expect that preventing untimely superintelligence will be much less expensive than World War 2 (understatement) this at least helps to make salient the possibility that states will forego many of the expected benefits of unrestricted AI development, for at least a while, even if that means a real impact to economic growth.

Looking Past the Threat

Coalitions only last so long as their threat does, and a common historical feature is that coalition members often direct their attention, during the conflict, to how events will unfold after the threat has been addressed. For example, the Bretton Woods conferences, held even while World War II was still being fought, created the global financial system required to prevent economic instability in the war’s aftermath. A successful, concerted global effort to safely navigate the advent of superintelligence will require similar (likely even more intense) effort. In this way, our plan is forward-looking and dynamic, and unlike other types of international arrangements which may be more preserving of the status-quo (consider nonproliferation or arms blimitations). Its objective is to create the time required for the study of the many problems which ASI presents, and the creation of solutions to those problems so that the halt can be concluded.

Limitations

Coalitions are themselves not perfect analogies for the agreement. One could reasonably object that the threat isn’t clear, external, or actually defeatable. I contend that the threat will become more clear as time progresses, and hopefully it will become clear enough for action in time. It is not an external threat in the style of a hostile state, though it does not take too much effort to imagine a “country of geniuses in a data center” as a hostile foreign entity, and one can also imagine the resounding defeat of such an adversary.

Comparing our proposal to defensive coalitions has highlighted several key properties: they are motivated by serious threats, they invoke material action, and they are inherently forward-looking. I will turn now to address the shortcomings of an oft-used analogy when considering plans like ours: nonproliferation.

Nonproliferation Agreement

Weapons of mass destruction present the potential for catastrophe, similar to advanced AI. Efforts to prevent the proliferation of weapons of mass destruction could be a key analogy for efforts to mitigate AI risk. Nuclear weapons, in particular, have been the focus of a mostly successful, legible, multilateral, and many-decades long nonproliferation effort which has limited their spread. Even though they were developed eighty years ago, proliferation has not occurred beyond a relatively small group of countries and nuclear weapons have been kept out of the hands of terrorists and others in the apocalyptic residual. This positive example inspires work like Superintelligence Strategy by Dan Hendrycks et al. Its section on nonproliferation aims to secure the inputs and outputs of the “AI pipeline” from rogue, non deterrable, actors. In this way, it is similar to the efforts like the Cooperative Threat Reduction (CTR) program which helped to secure and dismantle nuclear, chemical, and biological weapons and infrastructure in former Soviet states, to mitigate the risk that they would fall into the hands of madmen or terrorists.

Our agreement is different: it blocks the initial development of dangerous AI technology by anyone, anywhere. Not just madmen or terrorists but by superpowers and international corporations. In Superintelligence Strategy’s own terms, our agreement secures the inputs to AI development but also prohibits the creation of its outputs. We aim to stop the advancement of both AI capabilities and AI development know-how, rather than merely preventing them from falling into the wrong hands. This would give nonproliferation some defense-in-depth: creating ASI would require not just a covert operation to do the final training run, but also to conduct the research which precedes it in secret and while avoiding disruption.

Helen Toner describes the difficulty of nonproliferation of dangerous AI given the continuing advancement of frontier capabilities. We agree, and are not enthusiastic about avoiding the dangers of ASI in worlds where capabilities continue to advance anything like the way they are at present. But we contend that the advancement of frontier capabilities is not inevitable, and offer our plan as an example of how to do so feasibly, with consideration for both technical and political considerations.

With these limitations in mind, many of the techniques of nonproliferation are relevant to the implementation of the agreement. These include export controls on AI chips and semiconductor manufacturing equipment (SME), monitoring of compute production, distribution, and use, and the use of inducements and the threat of enforcement to gain compliance.

Conclusion

The development of artificial superintelligence represents a threat unlike any humanity has previously faced—one that could permanently end human civilization. In confronting this danger, we must think beyond conventional frameworks. What we propose most critically has the properties of a defensive coalition: a temporary but vital alliance born of necessity.

The path forward will not be easy. It requires the U.S. and China to recognize that their competition for technological supremacy pales in comparison to the risk of either of them, or anyone else, creating an unaligned superintelligence.

or at least enough fear among the concerned that such actors might exist ↩︎