Related: Ideas for AI labs: Reading list. See also: AI labs' statements on governance.

This document is about AI policy ideas. It's largely from an x-risk perspective. ~~Strikethrough~~ denotes sources that I expect are less useful for you to read.

Lists

Lists of government (especially US government) AI policy ideas. I recommend carefully reading the lists in the first ~5 bullets in this list, noticing ideas to zoom in on, and skipping the rest of this section.

Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims (Brundage et al. 2020)
- Followed up by Filling gaps in trustworthy development of AI (Avin et al. 2021)
12 tentative ideas for US AI policy (Muehlhauser 2023) (EAF)
Survey on intermediate goals in AI governance (Räuker and Aird 2023)
Policymaking in the Pause (FLI 2023) (LW)
Frontier AI Regulation: Managing Emerging Risks to Public Safety (Anderljung et al. 2023)
"30 actions to reduce existential risk" in Existential risk and rapid technological change (Stauffer et al. 2023)
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation (Brundage et al. 2018)
How major governments can help with the most important century (Karnofsky 2023) (LW)
[Chuck Schumer] (2023)
[Dan Hendrycks] (2023)
"Discussion" in "Verifying Rules on Large-Scale NN Training via Compute Monitoring" (Shavit 2023)
Future Proof (CLTR 2021)
~~Year 1 Report~~ ~~(National Artificial Intelligence Advisory Committee 2023)~~
~~Final Report~~ ~~(National Security Commission on Artificial Intelligence 2021)~~
~~Challenges to U.S. National Security and Competitiveness Posed by AI~~ ~~(Matheny 2023)~~
~~"Policy Options" in "Artificial Intelligence and Strategic Trade Controls"~~ ~~(Viski et al. 2020)~~
~~Existential and global catastrophic risk policy ideas database~~ ~~(filter for "Artificial intelligence") (Sepasspour et al. 2022)~~
Various private lists and works in progress

Levers

Some sources focus on policy levers rather than particular policy proposals.

"Governmental levers" in "Literature Review of Transformative AI Governance" (Maas draft)
- This report is excellent
AI Policy Levers: A Review of the U.S. Government's Tools to Shape AI Research, Development, and Deployment (Fischer et al. 2021)
"Affordances" in "Framing AI strategy" (Stein-Perlman 2023)
Current UK government levers on AI development (Hadshar 2023)
Standards (largely a non-government lever)
- How technical safety standards could promote TAI safety (O'Keefe et al. 2022)
- Standards for AI Governance (Cihon 2019)
- Actionable-guidance and roadmap recommendations for the NIST AI Risk Management Framework (Barrett et al. 2022) (LW)
- ~~AI Risk Management Framework~~ ~~(NIST 2023)~~
- ~~Ethically Aligned Design~~ ~~(IEEE 2017)~~
- ~~Global AI Standards Repository~~ ~~(OCEANIS)~~

Other policy guidance

Desiderata

Some sources focus on abstract desiderata rather than how to achieve them.

Ideas^[1]

Verifying Rules on Large-Scale NN Training via Compute Monitoring (Shavit 2023) (LW)
Why and How Governments Should Monitor AI Development (Whittlestone and Clark 2021)
Regulatory Markets: The Future of AI Governance (Hadfield and Clark 2023)
Why we need a new agency to regulate advanced artificial intelligence: Lessons on AI control from the Facebook Files (Korinek 2021)
Compute Funds and Pre-trained Models (Anderljung 2022)
AI & Antitrust: Reconciling Tensions Between Competition Law and Cooperative AI Development (Hua and Belfield 2021)
- See also Antitrust-compliant AI industry self-regulation (O'Keefe 2021)
- See also AI & antitrust/competition law (Aird 2022)
How We Can Regulate AI (Balwit 2023) [compute governance]
Regulating artificial intelligence: Proposal for a global solution (Erdélyi and Goldsmith 2022)
Immigration Policy and the Global Competition for AI Talent (Huang and Arnold 2020)
AI & Global Governance: Why We Need an Intergovernmental Panel for Artificial Intelligence (Miailhe 2018)
Liability
- New: LPP on liability
- Products liability law as a way to address AI harms (Villasenor 2019)
- Liability for artificial intelligence and other emerging digital technologies (European Commission 2019).
- Footnote 16 of Collective action on artificial intelligence: A primer and review (de Neufville and Baum 2021) points to academic work on AI and liability
Crafting Legislation to Prevent AI-Based Extinction (Cohen and Osborne 2023)
- This is technically a response to government request for comment, but is really a policy proposal
An AI Policy Tool for Today: Ambitiously Invest in NIST (Anthropic 2023)
- Or see the policy memo version of this blogpost
Sharing Powerful AI Models (Shevlane 2022)
You Can't Regulate What You Don't Understand (O'Rielly 2023)
Killer Robots Are Here—and We Need to Regulate Them (Trager and Luca 2022)
Incident reporting & tracking
- See PAI's AI Incidents Database (arXiv, blogpost)

Policy proposals in the mass media

AI's Gatekeepers Aren’t Prepared for What's Coming (FP: Scharre 2023)
- This piece is great
The world needs an international agency for artificial intelligence, say two AI experts (Economist: Marcus and Reuel 2023)
We Need a Manhattan Project for AI Safety (POLITICO: Hammond 2023)
Does the world need an arms control treaty for AI? (CyberScoop: Groll 2023)
AI Desperately Needs Global Oversight (WIRED: Chowdhury 2023)
The Surprising Thing A.I. Engineers Will Tell You if You Let Them (NYT: Klein 2023)
We Must Regulate A.I. Here’s How. (NYT: Khan 2023)
We Must Declare Jihad Against A.I. (Compact: Cuenco 2023)

Responses to government requests for comment

~~Response to the NTIA AI Accountability Policy~~ ~~(GovAI 2023)~~
~~GovAI Response to the Future of Compute Review - Call for Evidence~~ ~~(GovAI 2022)~~
~~Future of compute review - submission of evidence~~ ~~(CLTR et al. 2022)~~
~~Anthropic Comment Regarding "Study To Advance a More Productive Tech Economy"~~ ~~(Anthropic 2022)~~
~~National Security Addition to the NIST AI RMF~~ ~~(Special Competitive Studies Project 2023)~~
~~Reconfiguring Resilience for Existential Risk~~ ~~(CSER 2021)~~
~~Response to the UK's Future of Compute Review~~ ~~(CLTR et al. 2023)~~
~~Submission to the NIST AI Risk Management Framework~~ ~~(GovAI 2022)~~
~~Submission of Feedback to the European Commission's Proposal for a Regulation laying down harmonised rules on artificial intelligence~~ ~~(CSER 2021)~~
~~Advice to UN High-level Panel on Digital Cooperation~~ ~~(CSER and GovAI 2019)~~
~~Consultation on the European Commission's~~ ~~White Paper on Artificial Intelligence: a European approach to excellence and trust~~ ~~(GovAI 2020)~~

See also

Very non-exhaustive.

AI Governance: A Research Agenda (Dafoe 2018)
Slowing AI (Stein-Perlman 2023)
Survey on intermediate goals in AI governance (Räuker and Aird 2023)
FACT SHEET: Biden-⁠Harris Administration Announces New Actions to Promote Responsible AI Innovation that Protects Americans’ Rights and Safety (The White House 2023)
~~OECD AI Policy Observatory~~
~~PAI~~ ~~and the~~ ~~AI and Shared Prosperity Initiative~~
~~[Mark Warner]~~ ~~(2023)~~
~~EU AI Act~~ (EU)
~~CHIPS Act~~ (US)
~~A pro-innovation approach to AI regulation~~ ~~(UK Office for Artificial Intelligence 2023)~~

This post is largely missing policy levers beyond domestic policy: international relations,^[2] international organizations, and standards.

Some sources are roughly sorted within sections by a combination of x-risk-relevance, quality, and influentialness– but sometimes I didn't bother to try to sort them, and I've haven't read all of them.

Please have a low bar to suggest additions, substitutions, rearrangements, etc.

Thanks to Jakub Kraus and Seán Ó hÉigeartaigh for some sources.

Last updated: 10 July 2023.

^{^}
- I think these sources each focus on a particular policy idea– I haven't even skimmed all of them
- Very non-exhaustive
- Thanks to Sepasspour et al. 2022 and a private list for some sources
^{^}
International agreements seem particularly important and neglected.
One source (not focused on policy ideas or levers): Nuclear Arms Control Verification and Lessons for AI Treaties (Baker 2023).
Oliver Guest agrees that there are not amazing sources but mentions:
- CSET on international security in the context of military AI
- CNAS on international arms control and confidence-building measures for military AI