Concept extrapolation is the skill of taking a concept, a feature, or a goal that is defined in a narrow training situation... and extrapolating it safely to a more general situation. This more general situation might be very extreme, and the original concept might not make much sense (eg defining "human beings" in terms of quantum fields).
Nevertheless, since training data is always insufficient, key concepts must be extrapolated. And doing so successfully is a skill that humans have to a certain degree, and that an aligned AI would need to possess to a higher extent.
This sequence collects the key posts on concept extrapolation. They are not necessarily to be read in this order; different people will find different posts useful.
Model splintering: moving from one imperfect model to another is the original post on "model splintering" - what happens when features no longer make sense because the world-model has changed. A long post with a lot of overview and motivation explanations, showing that model splintering is a problem with almost all alignment methods.
Concept extrapolation is the skill of taking a concept, a feature, or a goal that is defined in a narrow training situation... and extrapolating it safely to a more general situation. This more general situation might be very extreme, and the original concept might not make much sense (eg defining "human beings" in terms of quantum fields).
Nevertheless, since training data is always insufficient, key concepts must be extrapolated. And doing so successfully is a skill that humans have to a certain degree, and that an aligned AI would need to possess to a higher extent.
This sequence collects the key posts on concept extrapolation. They are not necessarily to be read in this order; different people will find different posts useful.
Different perspectives on concept extrapolation collects many different analogies and models of concept extrapolation, intended for different audiences, and collected together here.
Model splintering: moving from one imperfect model to another is the original post on "model splintering" - what happens when features no longer make sense because the world-model has changed. A long post with a lot of overview and motivation explanations, showing that model splintering is a problem with almost all alignment methods.
General alignment plus human values, or alignment via human values? shows that concept extrapolation is necessary and almost sufficient for successfully aligning AIs.
Value extrapolation, concept extrapolation, model splintering defines and disambiguates key terms: model splintering, value extrapolation, and concept extrapolation.