Coding day in and out on LessWrong 2.0
Promoted to curated: It's been a while since this post has come out, but I've been thinking of the "credit assignment" abstraction a lot since then, and found it quite useful. I also really like the way the post made me curious about a lot of different aspects of the world, and I liked the way it invited me to boggle at the world together with you.
I also really appreciated your long responses to questions in the comments, which clarified a lot of things for me.
One thing comes to mind for maybe improving the post, though I think that's mostly a difference of competing audiences:
I think some sections of the post end up referencing a lot of really high-level concepts, in a way that I think is valuable as a reference, but also in a way that might cause a lot of people to bounce off of it (even people with a pretty strong AI Alignment background). I can imagine a post that includes very short explanations of those concepts, or moves them into a context where they are more clearly marked as optional (since I think the post stands well without at least some of those high-level concepts)
This post, and TurnTrout's work in general, have taken the impact measure approach far beyond what I thought was possible, which turned out to be both a valuable lesson for me in being less confident about my opinions around AI Alignment, and valuable in that it helped me clarify and think much better about a significant fraction of the AI Alignment problem.
I've since discussed TurnTrout's approach to impact measures with many people.
I've used the concepts in this post a lot when discussing various things related to AI Alignment. I think asking "how robust is this AI design to various ways of scaling up?" has become one of my go-to hammers for evaluating a lot of AI Alignment proposals, and I've gotten a lot of mileage out of that.
This post actually got me to understand how logical induction works, and also caused me to eventually give up on bayesianism as the foundation of epistemology in embedded contexts (together with Abram's other post on the untrollable mathematician).
I think this post, together with Abram's other post "Towards a new technical explanation" actually convinced me that a bayesian approach to epistemology can't work in an embedded context, which was a really big shift for me.
Promoted to curated: I think the strategy-stealing assumption is a pretty interesting conceptual building block for AI Alignment, and I've used it a bunch of times in the last two months. I also really like the structure of this post, and found it both pretty easy to understand, and to cover a lot of ground and considerations.
Do come visit our office in your basement sometimes.
Promoted to curated: This seems like it was a real conversation, and I also think it's particularly valuable for LessWrong to engage with more outside perspectives like the ones above.
I also in general want to encourage people to curate discussion and contributions that happen all around the web, and archive them in formats like this.
I often don't have much to say about these newsletters, since they usually only straightforwardly summarize things, or make statements that would take me a long time to engage with, but it seemed good to mention that this edition was particularly helpful to me (because I've been considering whether to invest the time to read all of the book, and this made it more likely that I will, since I seem to disagree with at least a bunch of the things you summarized here)
I felt a bit uncertain about doing one every month, and was planning to start another one in October. Depending on how that one goes we might go with a monthly schedule, or maybe every two months is the right way to go.