The LessWrong Annual Review is a
yearly event where posts from a previous year are nominated, reviewed, and voted on.central mechanism for reviewing the site's content, it's kind of like our site's peer review process.
OnLessWrong we're trying to makehas the goal of making intellectual progress on problemsimportant problems. To make progress, you gotta examine your community's outputs not only when they're first published, but also once enough time has passed to see whether they continued to provide value after initial hype fades and flaws have had time to surface.
So far, the bulk of the LessWrong Review happens in the winter season, over December and January. Batched yearly, we review the posts from the previous year. 2019's post get reviewed at the end of 2020 (i.e. every post in each review batch is 1-2 years old).
For two months, the community nominates, writes reviews, and votes on the most valuable posts. Reviewers ask questions like:
- Does this post continue to affect my thinking or actions even 12+ months later?
- Do its claims still seem valuable or interesting even now?
- Was it invalidated by further work or other criticisms that
matter, but itcame up?
- Which follow-up work do I still wish someone would do?
The community answering these questions leads to benefits like:
- We create broad knowledge about which ideas and results turned out to be robust and future work can be
hardbuilt upon (and which ideas failed to tell whether we're making progress.achieve that).
- Out of the thousands of posts published each year, we can direct the attention of future readers to the most valuable content.
- e.g. via the creation of Sequences, [e]books, etc.
- We can better reward and incentivize the thinkers whose contributions were most valuable.
[New!] The LessWrong team is currently experimenting with ways to give the Annual Review year-long salience, i.e. by giving people the ability to bet in prediction markets about which posts will be rated highly in the Review, and by giving Review "winners" prominent treatment (e.g. cool art on their post pages).
What is the Review process?
The exact process of the Annual Review is
the once-yearly Schelling timenot locked in stone and we might decide to reflect, debate, discuss and vote. It's a poke to say "remember that idea you and everyone else were obsessed with in 20xx? Didchange
On LessWrong we're trying to make intellectual progress on problems that matter, but it can be hard to tell whether we're making progress. The Annual Review is the once-yearly Schelling time to reflect, debate, discuss and vote. It's a poke to say "remember that idea you and everyone else were obsessed with in 20xx? Did it continue to matter or was it just a fad?" It's also the time when we can determine which contributions most affected people in the long-term, and reward those authors and inspire others to follow suit. I think it's pretty great that when an author who invested hours into a post and greatly benefits the community, the community eventually recognizes and rewards them in some fashion.
- Preliminary Voting Phase: Site members start identifying posts especially worthy of consideration in the review. Posts with 2 preliminary votes move into the Discussion Phase.
- Discussion Phase: Site members begin reviewing and debating posts in earnest. This discussion will inform members on how to cast their final votes. Posts that receive at least one written review move to the Final Voting phase.
- Final Voting: Site members allocate their vote budget (quadratic voting) among nominated posts. The outcome of the final determines the Annual Review results.