The Global Call for AI Red Lines was released and presented in the opening of the first day of the 80th UN General Assembly high-level week.

Maria Ressa, Nobel Peace Prize, announcing the call at the opening of the UN General Assembly: "We urge governments to establish clear international boundaries to prevent unacceptable risks for AI. At the very least, define what AI should never be allowed to do"

It was initiated by CeSIA, the French Center for AI Safety, in partnership with The Future Society and the Center for Human-compatible AI.

The full text of the call reads:

AI holds immense potential to advance human wellbeing, yet its current trajectory presents unprecedented dangers. AI could soon far surpass human capabilities and escalate risks such as engineered pandemics, widespread disinformation, large-scale manipulation of individuals including children, national and international security concerns, mass unemployment, and systematic human rights violations.
Some advanced AI systems have already exhibited deceptive and harmful behavior, and yet these systems are being given more autonomy to take actions and make decisions in the world. Left unchecked, many experts, including those at the forefront of development, warn that it will become increasingly difficult to exert meaningful human control in the coming years.
Governments must act decisively before the window for meaningful intervention closes. An international agreement on clear and verifiable red lines is necessary for preventing universally unacceptable risks. These red lines should build upon and enforce existing global frameworks and voluntary corporate commitments, ensuring that all advanced AI providers are accountable to shared thresholds.
We urge governments to reach an international agreement on red lines for AI — ensuring they are operational, with robust enforcement mechanisms — by the end of 2026.

This call has been signed by a coalition of 200+ former heads of state, ministers, diplomats, Nobel laureates, AI pioneers, industry experts, human rights advocates, political leaders, and other influential thinkers, as well as 70+ organizations.

Signatories include:

10 Nobel Laureates, in economics, physics, chemistry and peace
Former Heads of State: Mary Robinson (Ireland), Enrico Letta (Italy)
Former UN representatives: Csaba Kőrösi, 77th President of the UN General Assembly
Leaders and employees at AI companies: Wojciech Zaremba (OpenAI cofounder), Jason Clinton (Anthropic CISO), Ian Goodfellow (Principal Scientist at Deepmind)
Top signatories from the CAIS statement: Geoffrey Hinton, Yoshua Bengio, Dawn Song, Ya-Qin Zhang

Theory of Change

In Seoul, companies pledged to “Set out thresholds at which severe risks posed by a model or system, unless adequately mitigated, would be deemed intolerable”, but there is still nothing today that prevents Meta/xAI/everyone from setting thresholds too high, or not setting them at all. Without common rules, this race is a race to the bottom, and safety-conscious actors are going to be disadvantaged.

Why are international AI red lines important?

International AI red lines are critical because they establish clear prohibitions on the development, deployment, and use of systems that pose unacceptable risks to humanity. They are:

What are some examples of red lines?

The red lines could focus either on AI behaviors (i.e., what the AI systems can do) or on AI uses (i.e., how humans and organizations are allowed to use such systems). The following examples show the kind of boundaries that can command broad international consensus.

Note that the campaign does not focus on endorsing any specific red lines. Their specific definition and clarification will have to be the result of scientific-diplomatic dialogues.

Examples of red lines on AI uses:

Nuclear command and control: Prohibiting the delegation of nuclear launch authority, or critical command-and-control decisions, to AI systems (a principle already agreed upon by the US and China).
Lethal Autonomous Weapons: Prohibiting the deployment and use of weapon systems used for killing a human without meaningful human control and clear human accountability.
Mass surveillance: Prohibiting the use of AI systems for social scoring and mass surveillance (adopted by all 193 UNESCO member states).
Human impersonation: Prohibiting the use and deployment of AI systems that deceive users into believing they are interacting with a human without disclosing their AI nature.

Examples of red line on AI behaviors:

Cyber malicious use: Prohibiting the uncontrolled release of cyberoffensive agents capable of disrupting critical infrastructure.
Weapons of mass destruction: Prohibiting the deployment of AI systems that facilitate the development of weapons of mass destruction or that violate the Biological and Chemical Weapons Conventions.
Autonomous self-replication: Prohibiting the development and deployment of AI systems capable of replicating or significantly improving themselves without explicit human authorization (Consensus from high-level Chinese and US Scientists).
The termination principle: Prohibiting the development of AI systems that cannot be immediately terminated if meaningful human control over them is lost (based on the Universal Guidelines for AI).

Red lines on AI behaviors have already started being operationalized in the Safety and Security frameworks from AI companies such as Anthropic’s Responsible Scaling Policy, OpenAI’s Preparedness Framework, and DeepMind’s Frontier Safety Framework. For example, for AI models above a critical level of cyber-offense capability, OpenAI states that “Until we have specified safeguards and security controls standards that would meet a critical standard, halt further development.” Definitions of critical capabilities that require robust mitigations would need to be harmonized and strengthened between those different companies.

Are international AI red lines even possible?

Yes, history shows that international cooperation on high-stakes risks is entirely achievable. When the cost of inaction is too catastrophic, humanity has consistently come together to establish binding rules to prevent global disasters or profound harms to humanity and global stability.

The Treaty on the Non-Proliferation of Nuclear Weapons (1970) and the Biological Weapons Convention (1975) were negotiated and ratified at the height of the Cold War, proving that cooperation is possible despite mutual distrust and hostility. The Montreal Protocol (1987) averted a global environmental catastrophe by phasing out ozone-depleting substances, and the UN Declaration on Human Cloning (2005) established a crucial global norm to safeguard human dignity from the potential harms of reproductive cloning. Most recently, the High Seas Treaty (2025) provided a comprehensive set of regulations for high seas conservation and serves as a sign of optimism for international diplomacy.

In the face of global, irreversible threats that know no borders, international cooperation is the most rational form of national self-interest.

Who would enforce these red lines?

There is no single global authority for AI, so enforcement would likely combine different levels of governance, including:

International Treaty: A binding agreement would harmonize rules across countries. This prevents a regulatory arbitrage "race to the bottom" where companies could evade regulations by moving to less strict jurisdictions.
National Governments: Nations would be tasked with translating the international agreement into domestic law. This would involve creating regulatory bodies to license advanced AI systems, conduct mandatory safety audits, and impose severe penalties for violations within their jurisdictions.
International Technical Verification Body: An impartial international body, modeled on organizations like the International Atomic Energy Agency (IAEA), could develop standardized auditing protocols and independently verify that AI systems from any company or country comply with the agreed-upon red lines. The International Network of AI Safety and Security Institutes is well-positioned to play a role in this process.

What should the next steps be?

Several complementary pathways could be envisaged:

A group of pioneering countries or a “coalition of the willing,” potentially drawing on countries already engaged in the G7 Hiroshima AI process, could advance the concept of AI red lines across the G7, G20, and BRICS agendas.
The newly established UN Independent Scientific Panel on AI could publish a thematic brief articulating scientific consensus on technically clear and verifiable red lines, with technical contributions from the OECD.
Building on this groundwork, states could use the AI Impact Summit in India in February 2026 to endorse initial red lines for AI. Such red lines could build on the Seoul Commitments by translating voluntary corporate pledges into shared risk thresholds that, in turn, could be embedded in national regulation and public procurement.
The UN Global Dialogue on AI Governance could lead a global consultation with scientists, civil society, and industry to define a set of clear and verifiable red lines, to be summarized in an outcome document at the July 2026 Dialogue in Geneva.
By the end of 2026, either (i) a UN General Assembly resolution could be initiated, noting and welcoming these red lines and inviting negotiations, or (ii) a joint ministerial statement by an alliance of willing states could launch negotiations for a binding international treaty.

Any future treaty should be built on three pillars: a clear list of prohibitions; robust, auditable verification mechanisms; and the appointment of an independent body established by the Parties to oversee implementation.

You can access the website here: https://red-lines.ai

Happy to get feedback on the red lines agenda. CeSIA is currently considering continuing the work needed to ensure those red lines come to life, and there is still a lot of work ahead.

Thanks to Arthur Grimonpont, who drafted the first version of the FAQ. Thanks to Niki Iliadis, Su Cizem, and Tereza Zoumpalova for their significant revision. Thanks to Lucie Philippon for helping with the LW post. And thanks for the many people who worked hard to make this happen.