Jesse Hoogland

Neural networks generalize because of this one weird trick

> Produced under the mentorship of Evan Hubinger as part of the SERI ML Alignment Theory Scholars Program - Winter 2022 Cohort > A big thank you to all of the people who gave me feedback on this post: Edmund Lau, Dan Murfet, Alexander Gietelink Oldenziel, Lucius Bushnaq, Rob Krzyzanowski, Alexandre Variengen, Jiri Hoogland, and Russell Goyder. Statistical learning theory is lying to you: "overparametrized" models actually aren't overparametrized, and generalization is not just a question of broad basins. The standard explanation thrown around here for why neural networks generalize well is that gradient descent settles in flat basins of the loss function. On the left, in a sharp minimum, the updates bounce the model around. Performance varies considerably with new examples. On the right, in a flat minimum, the updates settle to zero. Performance is stabler under perturbations. To first order, that's because loss basins actually aren't basins but valleys, and at the base of these valleys lie "rivers" of constant, minimum loss. The higher the dimension of these minimum sets, the lower the effective dimensionality of your model.[1] Generalization is a balance between expressivity (more effective parameters) and simplicity (fewer effective parameters). Symmetries lower the effective dimensionality of your model. In this example, a line of degenerate points effectively restricts the two-dimensional loss surface to one dimension. In particular, it is the singularities of these minimum-loss sets — points at which the tangent vanishes — that determine generalization performance. The remarkable claim of singular learning theory (the subject of this post), is that "knowledge … to be discovered corresponds to singularities in general" [1]. Complex singularities make for simpler functions that generalize further. The central claim of singular learning theory is that the singularities of the set of minima of the loss function determine learning behavior and generalization.

204Jan 18, 2023

Jesse Hoogland

Message

Executive director at Timaeus. Working on singular learning theory and developmental interpretability.

Website: jessehoogland.com

Twitter: @jesse_hoogland

3159

875

SLT for AI Safety

> This sequence draws from a position paper co-written with Simon Pepin Lehalleur, Jesse Hoogland, Matthew Farrugia-Roberts, Susan Wei, Alexander Gietelink Oldenziel, Stan van Wingerden, George Wang, Zach Furman, Liam Carroll, Daniel Murfet. Thank you to Stan, Dan, and Simon for providing feedback on this post. Alignment ⊆ Capabilities. As...

Jul 1, 202576

The Sweet Lesson: AI Safety Should Scale With Compute

A corollary of Sutton's Bitter Lesson is that solutions to AI safety should scale with compute.[1] Let's consider a few examples of research directions that are aiming at this property: * Deliberative Alignment: Combine chain-of-thought with Constitutional AI to improve safety with inference-time compute (see Guan et al. 2025, Figure...

May 5, 202597

Timaeus in 2024

> TLDR: We made substantial progress in 2024: > > * We published a series of papers that verify key predictions of Singular Learning Theory (SLT) [1, 2, 3, 4, 5, 6]. > * We scaled key SLT-derived techniques to models with billions of parameters, eliminating our main concerns around...

Feb 20, 2025100

Timaeus is hiring researchers & engineers

TLDR: We're hiring for research & engineering roles across different levels of seniority. Hires will work on applications of singular learning theory to alignment, including developmental interpretability. About Us Timaeus' mission is to empower humanity by making breakthrough scientific progress on alignment. Our research focuses on applications of singular learning...

Jan 17, 202565

Building AI Research Fleets

From AI scientist to AI research fleet Research automation is here (1, 2, 3). We saw it coming and planned ahead, which puts us ahead of most (4, 5, 6). But that foresight also comes with a set of outdated expectations that are holding us back. In particular, research automation...

Jan 12, 2025132

o1: A Technical Primer

> TL;DR: In September 2024, OpenAI released o1, its first "reasoning model". This model exhibits remarkable test-time scaling laws, which complete a missing piece of the Bitter Lesson and open up a new axis for scaling compute. Following Rush and Ritter (2024) and Brown (2024a, 2024b), I explore four hypotheses...

Dec 9, 2024172

Timaeus is hiring!

TLDR: We’re hiring two research assistants to work on advancing developmental interpretability and other applications of singular learning theory to alignment. About Us Timaeus’s mission is to empower humanity by making breakthrough scientific progress on alignment. Our research focuses on applications of singular learning theory to foundational problems within alignment,...

Jul 12, 202467

Load More (7/18)

AI ALIGNMENT FORUM
AF

AI ALIGNMENT FORUM
AF

Jesse Hoogland

Jesse Hoogland

Jesse Hoogland

Neural networks generalize because of this one weird trick

Towards Developmental Interpretability

Announcing Timaeus

Timaeus's First Four Months

Jesse Hoogland

SLT for AI Safety

The Sweet Lesson: AI Safety Should Scale With Compute

Timaeus in 2024

Timaeus is hiring researchers & engineers

Building AI Research Fleets

o1: A Technical Primer

Timaeus is hiring!

Neural networks generalize because of this one weird trick

Towards Developmental Interpretability

Announcing Timaeus

Timaeus's First Four Months

SLT for AI Safety

The Sweet Lesson: AI Safety Should Scale With Compute

Timaeus in 2024

Timaeus is hiring researchers & engineers

Building AI Research Fleets

o1: A Technical Primer

Timaeus is hiring!