TL;DR: There are many posts on the Alignment Forum/LessWrong that could easily be on arXiv. Putting them on arXiv has several large benefits and (sometimes) very low costs.
Benefits of having posts on arXiv
There are several large benefits of putting posts on arXiv:
- 1. Much better searchability, shows up in google scholar searches.
- 2. Additional reads (arXiv sanity, arXiv newsletters, and so on).
- 3. The article can accumulate citations, which are shown in google/google scholar search results.
1) - 3) lead to more people reading your research, which hopefully leads to more people building on it and maybe useful feedback from outside of the established alignment community. In particular, if people see that the paper already has citations, this will lead to more people reading it, which will lead to more citations, and so on.
You'd gain even more of 2) and 3) from publishing it at a conference, but, unlike arXiv, that's significant additional work (often still worth it).
There are also some smaller benefits from publishing on arXiv:
- 4. firmly establishes this as your contribution (not sure, but I think if you only have an alignment forum post, someone could build a bit on it and then claim the whole thing as their contribution because alignment forum posts don't count?).
- 5. better citability (e.g. if somebody writes an ML paper to be published in ML venues, it gives more credibility to cite arXiv papers than Alignment Forum/LessWrong posts. The same goes for people writing e.g. Wikipedia articles about alignment.)
How much work is it to submit to arXiv?
Citing DavidHolmes from the comments: "There is a certain amount of moderation on arXiv. This is a little opaque [...]. In writing this I don't want to give the impression that posting things to arXiv is hard; I have currently 28 papers there, have never had a single problem or delay with moderation, and the submission process generally takes me <15 mins these days."
Sometimes, I think getting your forum post ready for submission can be as easy as creating a pdf of your post (although if your post was written in LaTeX, they'll want the tex file). If everything goes well, the submission takes less than an hour.
However, if your post doesn't look like a research article, you might have to format it more like one (and even then it's not guaranteed to get in, see this comment thread).
If you are submitting to arXiv for the first time, you might have to get an endorsement from someone who has already published on arXiv. The endorsement sometimes won't be required if you have an academic email address, so be sure to use that one for submission.
If you're a Very Busy Alignment Researcher, I'm sure you can outsource large parts of this. E.g. FAR's comms staff could probably help. I also have worked with a freelancer on similar things in the past (like making publications look nice in LaTeX), feel free to reach out for the contact data.
Highlights from the comments:
- DavidHolmes's tips for how to submit to arXiv.
- Dan Hendrycks's suggestions on which sections of arXiv are suitable for alignment work.
- This comment thread suggests that academic formatting/layout will probably be useful.
What types of posts should be on arXiv?
To be clear, I think that most LessWrong posts should not be on arXiv. The bar for submitting to arXiv should be higher than that for submitting to LessWrong/AF. Still, there are many research contributions on the Alignment Forum/LessWrong that wouldn't look out of place on arXiv. Submitting to arXiv is particularly useful if the post's target audience is wider than the LessWrong readership.
Here are some examples of posts that also are on arXiv, or, IMO, should be (skewed by what I read and remembered):
- Conceptual and theoretical research:
- Joe Carlsmith's Draft report on existential risk from power-seeking AI (it's on arXiv now, but >1 year after it was published)
- Ajeya Cotra's Draft report on AI timelines (not on arXiv but should be)
- Richard Ngo's AGI safety from first principles (not on arXiv but should be)
- I haven't read it, but from the name of it, it sounds as if ARC's first technical report: Eliciting Latent Knowledge should maybe be on arXiv
- The Truthful AI work by Lin, Evans, Cotton-Barratt, and others (conceptual paper on arXiv)
- Hendrycks's and Mazeika's X-risk Analysis for AI Research (on arXiv)
- Empirical ML research:
- High-stakes alignment via adversarial training [Redwood Research report] (on arXiv)
- The Truthful AI work by Lin, Evans, Cotton-Barratt, and others (empirical paper on arXiv)
- Neel Nanda's and Tom Lieberum's A Mechanistic Interpretability Analysis of Grokking (To the surprise of me and several others, this post was actually not accepted by arXiv.)
- Lots of ML alignment and safety work by Hendrycks et al., e.g. this, this, and this (all on arXiv)
- Language models seem to be much better than humans at next-token prediction could be on arXiv, but would probably need some edits to adhere more to "scientific paper style"
Highlights from the comments:
- Dan Hendrycks's suggestions on what to put (and not put) on arXiv.
- Davidmanheim with three more examples of alignment papers he put on arXiv.
If arXiv doesn't fit
There are also some other posts on the Alignment Forum/LessWrong whose target audience is the wider AI community. I think should be published additionally elsewhere. A great example is (one of my all-time favourite posts) Ajeya Cotra's Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover. This maybe wouldn't really fit on arXiv (although it wouldn't be crazy to put on on arXiv either). But there is a range of other venues that might publish it, from Towards Data Science (on the low effort, low prestige end) to the MIT Technology Review (on the high effort, high prestige end).