AI ALIGNMENT FORUM
AF

Superposition

Edited by duck_master last updated 5th Dec 2023

Posts about the concept of superposition - that is, neural nets representing concepts as a superposition of many neurons.

Posts tagged Superposition

69[Interim research report] Taking features out of superposition with sparse autoencoders

Lee Sharkey, Dan Braun, beren

3y

14

90Toward A Mathematical Framework for Computation in Superposition

Dmitry Vaintrob, jake_mendel, Kaarel

2y

8

55Circuits in Superposition: Compressing many small neural networks into one

Lucius Bushnaq, jake_mendel

11mo

1

33Circuits in Superposition 2: Now with Less Wrong Math

Linda Linsefors, Lucius Bushnaq

2mo

0

33Superposition is not "just" neuron polysemanticity

1y

1

22Some costs of superposition

Linda Linsefors

1y

8

110Towards Monosemanticity: Decomposing Language Models With Dictionary Learning

Zac Hatfield-Dodds

2y

12

56Comparing Anthropic's Dictionary Learning to Ours

2y

1

38Open Source Sparse Autoencoders for all Residual Stream Layers of GPT2-Small

2y

12

33Growth and Form in a Toy Model of Superposition

Liam Carroll, Edmund Lau

2y

0

32Interpretability with Sparse Autoencoders (Colab exercises)

CallumMcDougall

2y

0

33Toy Models of Superposition

3y

2

31Paper: Superposition, Memorization, and Double Descent (Anthropic)

3y

11

21Some open-source dictionaries and dictionary learning infrastructure

2y

4

18200 COP in MI: Exploring Polysemanticity and Superposition

3y

1

Load More (15/17)

Add Posts