Cullen

Law-Following AI 1: Sequence Introduction and Structure

This post is written in my personal capacity, and does not necessarily represent the views of OpenAI or any other organization. Cross-posted to the Effective Altruism Forum. This sequence of posts will argue that working to ensure that AI systems follow laws is a worthwhile way to improve the long-term future of AI.[1] The structure of this sequence will be as follows: * First, in this post, I will define some key terms and sketch what an ideal law-following AI ("LFAI") system might look like. * In the next few posts, I will explain why law-following might not emerge by default given the existing constellation of alignment approaches, financial objectives, and legal constraints, and explain why this is troubling. * Finally, I will propose some policy and technical routes to ameliorating these problems. If the vision here excites you, and you would like to get funding to work on it, get in touch. I may be excited to recommend grants for people working on this, as long as it does not distract them from working on more important alignment issues. Image by OpenAI's DALL·E. Key Definitions A law-following AI , or LFAI , is an AI system that is designed to rigorously comply with some defined set of human-originating rules ("laws"),[2] using legal interpretative techniques,[3] under the assumption that those laws apply to the AI in the same way that they would to a human. By "intrinsically motivated," I mean that the AI is motivated to obey those rules regardless of whether (a) its human principal wants it to obey the law,[4] or (b) disobeying the law would be instrumentally valuable.[5] (The Appendix to this post explores some possible conceptual issues with this definition of LFAI.) I will compare LFAI with intent-aligned AI. The standard definition of "intent alignment" generally concerns only the relationship between some property of a human principal H and the actions of the human's AI agent A: * Jan Leike et al. define the "agent alignment problem" as

18Apr 27, 2022

Cullen

Message

cullenokeefe.com

269

Tracking Compute Stocks and Flows: Case Studies?

Posted in my personal capacity The AGI governance community has recently converged on compute governance[1] as a promising lever for reducing existential risks from AI. One likely building block for any maximally secure compute governance regime is stock and flow accounting of (some kinds of) compute: i.e., requiring realtime accurate...

Oct 5, 202211

Law-Following AI 4: Don't Rely on Vicarious Liability

This post is written in my personal capacity, and does not necessarily represent the views of OpenAI or any other organization. Cross-posted to the Effective Altruism Forum. Image by OpenAI's DALL·E If an agent A causes some harm while intending to benefit a principal P, what is P's liability? The...

Aug 2, 20225

Law-Following AI 3: Lawless AI Agents Undermine Stabilizing Agreements

This post is written in my personal capacity, and does not necessarily represent the views of OpenAI or any other organization. Cross-posted to the Effective Altruism Forum In the previous post of this sequence, I argued that intent-aligned AIs would, by default, have incentives to break the law. This post...

Apr 27, 20222

Law-Following AI 2: Intent Alignment + Superintelligence → Lawless AI (By Default)

This post is written in my personal capacity, and does not necessarily represent the views of OpenAI or any other organization. Cross-posted to the Effective Altruism Forum. In the first post of this sequence, I defined "law-following AI" ("LFAI") and "intent alignment." In this post, I will begin to motivate...

Apr 27, 20225

Law-Following AI 1: Sequence Introduction and Structure

Apr 27, 202218

AI Benefits Post 5: Outstanding Questions on Governing Benefits

This is the final post in a series on "AI Benefits." It is cross-posted from my personal blog. For other entries in this series, navigate to the AI Benefits Blog Series Index page. This post is also discussed on the Effective Altruism Forum. For comments on this series, I am...

Jul 21, 20204

Parallels Between AI Safety by Debate and Evidence Law

In this post, I highlight some parallels between AI Safety by Debate (“Debate”) and evidence law. Evidence law structures high-stakes arguments with human judges. The prima facie reason that Evidence law (“Evidence”) is relevant to Debate is because Evidence is one of the few areas, like Debate, where debates have...

Jul 20, 202010

Load More (7/11)

AI ALIGNMENT FORUM
AF

AI ALIGNMENT FORUM
AF

Cullen

Cullen

Cullen

Law-Following AI 1: Sequence Introduction and Structure

Tracking Compute Stocks and Flows: Case Studies?

AI Benefits Post 1: Introducing “AI Benefits”

Parallels Between AI Safety by Debate and Evidence Law

Cullen

Tracking Compute Stocks and Flows: Case Studies?

Law-Following AI 4: Don't Rely on Vicarious Liability

Law-Following AI 3: Lawless AI Agents Undermine Stabilizing Agreements

Law-Following AI 2: Intent Alignment + Superintelligence → Lawless AI (By Default)

Law-Following AI 1: Sequence Introduction and Structure

AI Benefits Post 5: Outstanding Questions on Governing Benefits

Parallels Between AI Safety by Debate and Evidence Law

Law-Following AI 1: Sequence Introduction and Structure

Tracking Compute Stocks and Flows: Case Studies?

AI Benefits Post 1: Introducing “AI Benefits”

Parallels Between AI Safety by Debate and Evidence Law

Tracking Compute Stocks and Flows: Case Studies?

Law-Following AI 4: Don't Rely on Vicarious Liability

Law-Following AI 3: Lawless AI Agents Undermine Stabilizing Agreements

Law-Following AI 2: Intent Alignment + Superintelligence → Lawless AI (By Default)

Law-Following AI 1: Sequence Introduction and Structure

AI Benefits Post 5: Outstanding Questions on Governing Benefits

Parallels Between AI Safety by Debate and Evidence Law