In Selective regularization for alignment-focused representation engineering, we presented a successful approach for structuring the latent space of a simple MLP. Here we document our side quests: experiments that didn't go as expected, but in which we gained experience in regularization design and training dynamics.

This is the second part of Milestone 1 of Concept-anchored representation engineering for alignment. In this post, "we" refers to "me and Claude Opus 4," who helped draft this content. The underlying research was conducted with coding assistance from Claude 3.5 Sonnet, Gemini 2.5 Pro, GPT-4o, and GPT-4.1.

Lesson 1: Data curricula are mostly harmful

We initially believed curriculum learning in pre-training would help impose structure on latent spaces: we would start with key concepts to establish a foundation, then gradually add complexity. We thought this would be necessary to create a structure around which other concepts could organize, but our experiments ultimately revealed the opposite: training on the full dataset from the start produces better representations and more stable training dynamics^[1].

In our experiments, we trained a bottlenecked autoencoder on RGB colors. The primary objective was to reproduce the input color, but we added regularization to the bottleneck^[2] to make the network internally represent color as HSV instead of RGB^[3].

Following traditional curriculum learning principles, we started with just primary and secondary colors (6 points), then expanded to all hues, and finally introduced variations in brightness and saturation^[4]. We added an "anchor" to lock in the positions of key colors after the initial phase of the curriculum: once the primary and secondary colors stabilized, a regularizer kicked in to penalize drift in those specific colors. We expected that a clean color wheel would be discovered from just the primary and secondary colors, then gradually adding more colors should preserve that structure while filling in the gaps. It worked to some extent, but it had problems:

About 25% of training runs resulted in "crumpled" latent spaces where colors were out of order. The structure was therefore highly sensitive to initial conditions and hyperparameter choices.
When expanding the data to include previously out-of-bounds colors, new samples needed to be gently introduced (with low batch weight or learning rate) to

...