This is a special post for quick takes by TurnTrout. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
When talking about "self-fulfilling misalignment", "hyperstition" is a fun name but not a good name which actually describes the concept to a new listener. (In this sense, the name has the same problem as "shard theory" --- cool but not descriptive unless you already know the idea.) As a matter of discourse health, I think people should use "self-fulfilling {misalignment, alignment, ...}" instead.
"Hyperstition" has the benefit of grouping "self-fulfilling {misalignment, alignment, ...}" together though. It's plausible I'm totally wrong here, but I think "self-fulfilling misalignment" being the commonly used term lends itself to people thinking of the "answer" being data filtering of alignment discourse, while I want people to think about things like upsampling positive data even more (which "self-fulfilling alignment" does get at, but I hear the term much less and I think people will end up just indexing on the name they hear most).
I agree hyperstition isn't the best name for the reasons you describe to be clear, I just think "self-fulfilling {misalignment, ...}" also has a problem.
3TurnTrout
I think the problem of "may suggest a potentially suboptimal intervention" is less severe than "isn't descriptive." Plus, I think we're going to see "self-fulfilling alignment" be upsampled after the recent positive results. :)
Yeah, that's fair. I guess my views on this are stronger because I think data filtering might be potentially negative rather than potentially sub-optimal.