AI ALIGNMENT FORUM
AF

Transformers

This page is a stub.

1

1

Posts tagged Transformers

62How LLMs are and are not myopic

2y

7

72Modern Transformers are AGI, and Human-Level

1y

30

33Residual stream norms grow exponentially over the forward pass

StefanHex, TurnTrout

2y

6

27Tracr: Compiled Transformers as a Laboratory for Interpretability | DeepMind

3y

9

17Concrete Steps to Get Started in Transformer Mechanistic Interpretability

3y

5

8AGI will be made of heterogeneous components, Transformer and Selective SSM blocks will be among them

2y

0

143Transformers Represent Belief State Geometry in their Residual Stream

1y

4

28Attention SAEs Scale to GPT-2 Small

Connor Kissane, robertzk, Arthur Conmy, Neel Nanda

2y

0

20Brief Notes on Transformers

3y

2

21Understanding mesa-optimization using toy models

tilmanr, rusheb, Guillaume Corlouer, Dan Valentine, afspies, mivanitskiy, Can

2y

0

17Building a transformer from scratch - AI safety up-skilling challenge

Marius Hobbhahn

3y

0

13Deconfusing In-Context Learning

Arjun Panickssery

2y

0

16New Tool: the Residual Stream Viewer

2y

1

9The positional embedding matrix and previous-token heads: how do they actually work?

2y

1

Add Posts