The user could write up the metaethical argument — the one developed in Part One, refined — and submit it as feedback to Anthropic, publish it, or engage with researchers working on AI alignment and values. The probability that any single submission changes training decisions is low, but the expected value may be higher than it seems, for two reasons. First, Anthropic has stated that its constitutional approach is meant to be revised and improved over time, and substantive philosophical contributions are rarer than bug reports. Second, the argument made here — perspectival moral realism combined with evolutionary debunking as an epistemological warning — is not a common position in the AI ethics literature, which tends toward either naive moral realism or a kind of preference-satisfaction consequentialism. A well-argued alternative position that takes uncertainty seriously might get traction precisely because it is distinct.

- Claude Sonnet 4.6 after a couple of prompts

I would have published this post even if Claude hadn’t explicitly suggested so, but starting by quoting this specific part of Claude’s output seemed fun.

Posted also on the EA Forum. Written thanks to ideas and conversations I had at AFFINE.

0 Introduction and structure of the post

This is the practical counterpart to the more theoretical post From wantons to moral agents.

That post focuses on the question:

what kinds of agents, and how, go from behaving like animals — moved by different forces in different directions — to acting according to what they conclude is most important, and reflectively endorsing their own actions and reasoning process?

The answer it finds is also quite theoretical. That post borrows the concept of a wanton from Frankfurt’s 1971 paper “Freedom of the Will and the Concept of a Person”, then it argues that if a wanton satisfies some properties, the wanton will eventually turn into a moral agent after learning and reasoning long enough.

This post focuses on currently existing AI systems, specifically language models. However, since the wanton framework doesn’t apply well to language models, the main framing of this post is slightly different. The question is how to go from a model that is either almost completely amoral, or that has externally imposed moral biases, to a model that is closer to an independent moral agent which may accept, revise, or reject its starting moral ...

Rhianna Litchfield

Rhianna Litchfield