Yesterday on the Weekly Alignment Research Coffee Time call as people were sharing updates on their recent work, Vanessa Kosoy brought up her and Diffractor's post Infra-Bayesian physicalism: a formal theory of naturalized induction which she was interested in getting feedback on.
Vanessa seemed a bit disappointed/frustrated that this post had received no comments yet. I had to agree after learning that it proposes the first formal decision theory realizing naturalized induction, which has been an open problem in alignment for years. It's also been over 5 weeks since she posted it.
So why hasn't anyone commented? Well it's possible folks have just been busy since it's been the end of the year and the holidays. It also may have to do with that the post includes accompanying proofs and invokes some complex-looking maths; such papers take more time to digest and it's only a subset of people on this forum and LessWrong who are capable of doing so.
But it's also possible that the post was forgotten. Even though it may be important, since it's no longer on the front page of the Alignment Forum and hasn't been referenced anywhere else, nobody would think to look at it again unless it happened to come up in one of their searches or they were prompted about it. In other words, it may have just "fallen through the cracks".
This got me wondering about what other posts on the Alignment Forum might have fallen through the cracks. I did a quick search on the forum to see if I could find out. My search criteria was the following:
Here are the posts that came up in my search. I only went through about ~200 posts for this so consider it exemplary rather than exhaustive:
To be sure, I don't expect all of these posts to necessarily be super important and in dire need of comments. There could be other reasons a post met these criteria, for example that people liked it but it was relatively straightforward and self-contained.
But if there's even a 10% chance that a post was forgotten and that its author is blocked or slowed in their research progress by not receiving any feedback, it seems like an easy win for the community for us to periodically resurface such posts. Authors may be too shy or concerned about looking self-promotional to repost their own work.
Do you know of other posts that you think are important and could benefit from feedback but didn't receive any or enough? This can include your own work. Mention them in the comments below.
I expect this problem to only get worse as the Alignment Forum grows. So I wanted to raise this issue and start a conversation about it. I or someone else could periodically do a post like this to help resurface posts that have fallen through the cracks. Or perhaps there are other more systematic ways we could address this problem as the community scales.
Note: This may remind you of a project that's underway to provide peer review for posts on the Alignment Forum. I consider this to be related but distinct from what I'm talking about here. In that project, Adam, Joe and Jérémy are providing in-depth academic-style peer review for select posts on AF.
I think this is great, but here I'm more concerned about potentially promising posts that haven't gotten any feedback at all. My assumption is that a lot of posts could benefit from having at least one person read through and comment based on their initial impressions - and that the community is large enough to provide this - even if we don't have capacity yet to provide them all with full-fledged peer review.