(Apologies for this post being quite late – I got sick just prior to the holidays, and then had a surprisingly eventful holidays)

We've wrapped up the nominations phase for the 2021 Review. 139 people have participated so far, casting 1815 votes and writing 72 reviews.

Now we're in the Review Phase, where we ask "Were these posts actually good? How could they be improved? How do they fit together?".

Key Info:

Posts need at least one review in order to progress to the final voting stage. To write a review, navigate to a nominated post page, and click the "Review" button at the top of the page.
I'll award prizes to reviews that I think offer substantive information. Roughly $50 for a (See "What kinds of reviews are most valuable?" for examples)
You can look over the results of last year's review at /reviewVoting/2020, to compare the results there to this year's.
You can sort the posts on the /reviewVoting page by "Vote Total", to see which posts were most upvoted during the nomination voting. Thorough reviews of the top-scoring posts are particularly valuable.

How much Progress Overall?

One of my original goals for the Review was to provide a clearer feedback mechanism to the Lightcone team on "how is LessWrong doing?". It's one thing to track karma and site activity. But did our infrastructure maintenance, UI improvements, research retreats and other interventions actually seem to translate into more good, quality intellectual progress?

This year some people have commented that there are fewer exciting-feeling posts than they remember from previous years. I've also had a bit of this feeling – despite feeling at the time like in 2021, LessWrong was a pretty happening place with lots of good content. I'm not sure I endorse the feeling. For example: I think important progress happened in the MIRI Dialogues, but in a diffuse way, which didn't result in many concrete posts I can point at to that I learned discrete concepts from. Progress != posts, necessarily.

But, it still seems worth paying attention to the sensation. Yes requires the possibility of no, and it's fairly important for the Review to be able to return the answer "less overall progress this year."

Now, right now there isn't much of a mechanism for tracking "how much progress" there was between years. I'm still thinking about how to seriously answer this question. But for now, I've update the /reviewVoting page to be able to show previous years, so you can go eyeball the results from the previous 2020 Review, see how many posts you voted as a "9", etc.

What kinds of reviews are most valuable?

Briefly noting what you liked or didn't like about a post is good. But here are some specific things I'm excited for:

New Information. My favorite reviews are ones that give new information to the reader. This could be a concrete fact, or a rebuttal to a key argument in a post, or a new way the post is relevant that you hadn't considered. It could be a new frame about how to even think about the post.

Concrete use cases. I think reviews that say "this concretely helped me, here's how" are also great, especially if they give specifics. I think it's both helpful and rewarding to individual authors to hear how their post was useful, so they can do more of that. (I think this is also helpful on a broader level, so that the collective LW userbase can see what kinds of effects are realistic)

i.e. rather than saying "this post has helped me a bunch over the years", say "here are two specific times that it helped me, and how."

Thoughts on how the post could be improved. Two years later, if a post still seems like good reference material, but is confusing or poorly argued, give advice on how to improve it. Discuss the use-case for the readership you have in mind.

Reflect on the Big Picture. How do various posts fit together into something greater than the sum of their parts? What major conversations have happened on LessWrong and what have you taken from them?

Brevity/clarity. Bonus points if they are concise and relevant enough that it's fairly easy to print them in the 2021 book. (I wanted to print some reviews in the 2020 books but found it'd require a fair amount of editing work to get them into a shape that made sense). Note, I'm not saying "optimize for shortness at the expense of saying anything substantive", just, note that all-else-being-equal, taking less space to convey the key information is helpful.

Go Forth, Review, and Get Prizes

Remember, I'll be awarding prizes to reviews that I think add substantial new information, critique, or useful frames. Prizes will be announced in the comments here.