Basic Facts about Language Model Internals
This post was written as part of the work done at Conjecture. As mentioned in our retrospective, while also producing long and deep pieces of research, we are also experimenting with a high iteration frequency. This is an example of this strand of our work. The goal here is to...