Constitutional AI is a method for fine-tuning language models, used in Anthropic's Claude. The main conceptual difference from RLHF is that instead of human feedback on specific behaviors it relies on the model's ability to apply general principles (stated in natural language) to specific situations.