This is a special post for quick takes by TurnTrout. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
It's really important to use neutral, accurate terminology. AFAICT saying ML "selects for" low loss is unconventional. I think MIRI introduced this terminology. And if you have a bunch of intuitions about evolution being relevant to AI alignment and you want people to believe you on that, you can just use the same words for both optimization processes. Regardless of whether the processes share the relevant causal mechanisms, the two processes both "select for"stuff! They can't be that different, right?
And now the discussion moves on to debating the differences between two assumed-similar processes—does ML have less of an "information bottleneck"? Does that change the "selection pressures"? Urgh.
I think this terminology sucks and I wish it hadn't been adopted.
Be careful with words. Words shape how you think and your implicit category boundaries. When thinking privately, I often do better by tossing words to the side and thinking in terms of how each process works, and then considering what I expect to happen as a result of each process.
EDIT: In AGI Ruin: A List of Lethalities, Eliezer says that evolution had a "loss function" when he portrayed ML as similar to evolution:
See, again, the case of human intelligence. We didn't break alignment with the 'inclusive reproductive fitness' outer loss function
And (I don't immediately know where the comment is), gwern has described evolution as having a "reward function." This is sloppy shorthand at best, and misleading + distinction-blurring at worst.
Actually, I agreed too quickly. Words are not used in a vacuum. Even though this method isn't related to evolution, and even though a naive person might call it "selection" (and have that be descriptively reasonable), that doesn't mean it's best described as "selection." The reason is that the "s-word" has lots of existing evolutionary connotations. And on my understanding, that's the main reason you want to call it "selection" to begin with—in order to make analogical claims about the results of this proces... (read more)
It depends on what I'm trying to communicate. For example:
"ML selects for low loss" -> "Trained networks tend to have low training loss"
This correctly highlights a meaningful correlation (loss tends to be low for trained networks) and alludes to a relevant mechanism (networks are in fact updated to locally decrease loss on their training distributions). However, it avoids implying that the mechanism of ML is "selection on low loss."
3TurnTrout
Sure. But I think that's best described as "best-of-k sampling", which is still better because it avoids implicitly comparing selection-over-learning-setups (i.e. genotypes) with selection-over-parameterizations.
But let's just say I concede this particular method can be non-crazily called "selection." AFAICT I think you're arguing: "There exist ML variants which can be described as 'selection'." But speculation about "selecting for low loss" is not confined to those variants, usually people just lump everything in as that. And I doubt that most folks are on the edge of their seats, ready to revoke the analogy if some paper comes out that convincingly shows that ML diverges from "selecting for low loss"...[1]
1. ^
To be clear, that evidence already exists.