I'm seeing more sophisticated LLM-slop in the LW moderation queue.

Eight months ago, I wrote "hey, we're getting tons of AI-psychosis'd people, deluded into thinking their crackpot coherence/spiralism/emergence/ChatGPTAwakening experience is true and meaningful. We process like 15-20 of these a day."

Nowadays, we still get some of those, but, a lot less. Instead, I think we often now get a somewhat more sophisticated looking set of AI LLM slop. It's often something like:

"Me and ChatGPT have been working on some ML experiments for months, checking if we get some behavior if we do ablation, or if we mess with the residual stream, or, [some other mech-interp-ish buzzword thing]." They sound plausible at a glance, but usually don't explain the specific mechanism for why their experiment should be interesting, or fit into the LW conversation.

Often this person disclaims "I'm not an expert in ML, I have a degree in [X], but, I've done what seems like it should be a reasonable good faith effort to check if this makes sense."

I already find this fairly annoying to judge (a couple of them prompted disagreement within the team on whether they were legit). But, it seems like in another 6-12 months these will be fairly costly to distinguish from, say, a newly minted MATS scholar's work. And, maybe another 6-12 months after that, it might actually be better than many newly-minted-MATs-scholar's work.

(I think it will not reach the standard of a "sufficiently-good-alignment-researcher to really build meaningful conceptual progress" until Near The End. My impression is most junior MATS-scholars do work that's more like, executing on a shovel ready project where the judgment was done by their senior mentor and most of what they're doing is learning the ropes of how to run experiments. Interested in takes about this)

I'm really not sure what we're supposed to do about this.

My understanding of habryka's take is, he's guessing that 6-18 months from now, AI will actually be good enough at writing that it will feel about as silly for most people to do their writing/thinking unassisted, as it feels now to set up a new codebase and implement most straightforward features by end. And, if we want LW to be a place that's competitive and that new good thinkers want to participate in, it needs to somehow allow for that.

The new editor we're working on has "AI-written-section" as a first class type of paragraph block, with an intended norm that all AI content needs to go in said blocks. (expecting that posts that are entirely-AI-content-blocks usually won't meet our quality standards and get downvoted and/or mod-delisted on a case-by-case basis).

I find myself pretty worried about second-order effects on culture if we're going this route. (i.e. LW gets frogboiled by slop-thinking that's sophisticated enough that it gets upvoted by people who are half-paying attention at lunch, and then new users absorb that culture + incentive curve).

I don't have a particular takeaway or argument here, but, wanted to post the state of my confusion about it publicly and get some takes.