Geometry of Neural Activations

Planned topics include the linear representation hypothesis, computation in superposition, linear probes, sparse autoencoders (SAEs), and feature manifolds.

Add lecture notes, readings, and transcripts here as the course progresses.

Further reading

• Toy Models of Superposition — studies how neural networks represent more features than they have dimensions.
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learning — applies sparse feature decomposition to language models.
• Sky sphere representation in language models — preprint by Yevgeny Liokumovich and A. Berdnikov, listed on the instructor's page; relevant to geometric representations in language models.