Yeah, this all sounds right, and it's fairly close to the narrative I was using for my previous draft, which had a section on some of these motives.

The best defense I can give of the switch to the hype-centric framing, FWIW:

• The paper is inevitably going to have to do a lot of chastising of authors. Giving the most charitable possible framing of the motivations of the authors I'm chastising means that I'm less likely to lose the trust/readership of those authors and anyone who identifies with them.
• An increasingly large fraction of NLP work—possibly even a majority now—is on the analysis/probing/datasets side rather than model development, and your incentives 1 and 2 don't apply as neatly there. There are still incentives to underclaim, but they work differently.
• Practically, writing up that version clearly seemed to require a good deal more space, in an already long-by-ML-standards paper.

That said, I agree that this framing is a little bit too charitable, to the point of making implausible implications about some of these authors' motives in some cases, which isn't a good look. I also hadn't thought of the wasted effort point, which seems quite useful here. I'm giving a few talks about this over the next few weeks, and I'll workshop some tweaks to the framing with this in mind.

Forum  

I can see the comment at the comment-specific AF permalink here:

https://www.alignmentforum.org/posts/RLHkSBQ7zmTzAjsio/nlp-position-paper-when-combatting-hype-proceed-with-caution?commentId=pSkdAanZQwyT4Xyit#pSkdAanZQwyT4Xyit

...but I can't see it among the comments at the base post URL here.

https://www.alignmentforum.org/posts/RLHkSBQ7zmTzAjsio/nlp-position-paper-when-combatting-hype-proceed-with-caution 

From my experience with the previous comment, I expect it'll appear at the latter URL once I reply?

[Old technique] had [problem]...

Ah, got it. That makes sense! I'll plan to say a bit more about when/how it makes sense to cite older evidence in cases like this.

Thanks! Tentative rewrite for the next revision:

It harms our credibility in ways that can make it harder to mitigate present-day harms from NLP deployments. It also limits our ability to prepare for the potentially enormous impacts of more distant future advances.

I tried to stick to 'present-day' over 'short-term', but missed this old bit of draft text in the abstract. 

Thanks! (Typo fixed.)

[Old technique] had [problem]...

For this point, I'm not sure how it fits into the argument. Could you say more?

Is there any empirical base...

Yeah, this is a missed opportunity that I haven't had the time/expertise to take on. There probably are comparable situations in the histories of other applied research fields, but I'm not aware of any good analogies. I suspect that a deep dive into some history-and-sociology-of-science literature would be valuable here.

What if the impact grows dramatically as...they get deployed widely? ...

I think this kind of discussion is already well underway within NLP and adjacent subfields like FaCCT. I don't have as much to add there.

(Weird meta-note: Are you aware of something unusual about how this comment is posted? I saw a notification for it, but I didn't see it in the comments section for the post itself until initially submitting this reply. I'm newish to posting on Lightcone forums...)

Thanks—fixed! (The sentence-final period got folded into the URL.)

Another very minor (but briefly confusing) nit: The notation in the `Example' section is inconsistent between probabilities and log probabilities. It introduces $$H^{prior}\left(z\right)$$ (etc.) as a probability, but then treats it as a log probability in the line starting with 'We find the $z^{\ast }$'.