TLDR; This is the fourth main post of Distilling Singular Learning Theory which is introduced in DSLT0. I explain how to relate SLT to thermodynamics, and therefore how to think about phases and phase transitions in the posterior in statistical learning. I then provide intuitive examples of first and second order phase transitions in a simple loss function. Finally, I experimentally demonstrate phase transitions in two layer ReLU neural networks associated to the node-degeneracy and orientation-reversing phases established in DSLT3, which we can understand precisely through the lens of SLT.

In deep learning, the terms "phase" and "phase transition" are often used in an informal manner to refer to a steep change in a metric we care about, like the training or test loss, as a function of SGD steps, or alternatively some hyperparameter like the number of samples from the truth $n$ .

A classic example of a phase transition in deep learning literature is Anthropic's Induction Heads paper, where the "bump" in loss (in-context learning score) is referred to as a "phase change".

But what exactly are the phases? And why do phase transitions even occur? SLT provides us a solid theoretical framework for understanding phases and phase transitions in deep learning. In this post, we will argue that in the Bayesian setting,

A phase of the learning process corresponds to a singularity of $K (w)$ , and a phase transition corresponds to a drastic change in the posterior as a function of a hyperparameter $θ$ .

The hyperparameter $θ$ could be the number of samples from the truth $n$ , some way of varying the model function $f (x, w) = f (x, w; θ)$ or something about the true distribution $D_{n} = D_{n} (θ)$ , amongst other things. At some critical value $θ = θ_{c}$ , we recognise a phase transition as being a discontinuous change in the free energy or one of its derivatives, for example the generalisation error $G_{n} = E [F_{n + 1}] - E [F_{n}]$ .

In this post, we will present experiments that observe precise phase transitions in the toy neural network models we studied in DSLT3, for which we understand the set of true parameters $W_{0}$ and therefore the phases. By the end of this post, you will have a framework for thinking about phase transitions in singular models and an intuition for why SLT predicts them to occur in learning.

Phases Correspond to Singularities

The Story Starts in Physics

This subsection is modelled on [Callen, Ch9], but it is only intended to be a high level discussion of the concepts grounded in some basic physics - don't get too bogged down in the details of the thermodynamics.

Fundamentally, a phase describes an aggregate state of a complex system of many interacting components, where the state retains particular qualities with variations in some hyperparameter. To explain the concept in detail, it is natural to start in physics (thermodynamics in particular), where these ideas originally arose. But there is a deeper reason to build from here: every human has an intuitive understanding of the phases of water and how they change with temperature ^[1], which serves as the base mental model for what a phase is.

One of the main goals of thermodynamics is to study how the equilibrium state of a system changes as a function of macroscopic parameters. In the case of a vessel of water at 1atm of pressure in constant contact with a thermal and pressure reservoir, the equilibrium state of the system corresponds to a state that is minimised by the Gibbs free energy $F$ ^[2]. The phases, then, are the equilibrium states, which describe qualitative physical properties of the system. The states of matter - solid, liquid, and gas - are all phases of water, which are characterised by variables like their volume and crystal structure. As anybody that has boiled water before knows, these phases undergo transitions as a function of temperature. Let's make this more precise.

The Thermodynamic Setup

Consider a system of $K$ water molecules moving in a 2D container, each with equal mass $m$ . To each particle $i \in [K] = {1, \dots, K}$ we can associate a set of microstates describing its physical properties at a point in time, for example its position $x_{i}$ and its velocity $v_{i}$ . In our discussion we will simply focus on the position, which we will relabel $w = (w_{1}, \dots, w_{K})$ (for reasons that will become clear), so our configuration space $W \subseteq R^{2 K}$ of possible microstates is

W = {w | (w_{i, x}, w_{i, y}) \in R^{2} for each i \in [K]} .

Since it is physically infeasible to know or model the positions of all molecules, we instead reason about the dynamics of the system by calculating macroscopic variables associated to a microstate, for example the temperature or total volume of the molecules. We will focus on the volume $V (w)$ of a microstate $w$ . Importantly, a macroscopic state is an aggregate over the system (for example, temperature being related to average squared velocity), meaning there are many possible configurations of microstates that result in the same macrostate. To this end, we can define regions of our configuration space according to their volume $v$ ,

W_{v} = {w \in W | V (w) = v} \subseteq W .

In our toy example, we want to study how the system changes as a function of temperature, which we will denote with $θ$ . In a Gibbs ensemble, we can associate an energy functional, the Hamiltonian $H (w; θ)$ , to any given microstate $w$ at temperature $θ$ . The fundamental postulate of such a Gibbs ensemble is that probability of the system being in a particular micro state $w$ is determined by a Gibbs distribution ^[3]

p (w; θ) = \frac{e^{- H (w; θ)}}{Z} where Z = \int_{W} e^{- H (w; θ)} d w .

This should look pretty familiar from our statistical learning setup! Indeed, we can then calculate the free energy of the ensemble for different volumes $v$ at temperature $θ$ ,

F_{θ} (v) = - log (\int_{W_{v}} e^{- H (w; θ)} d w) .

For a Gibbs ensemble, the equilibrium state of a given system is that state which minimises the free energy. In the context of bringing water to a boiling point, there are two minima of the free energy characterised by the liquid and gaseous states, which for ease we will characterise by their volumes $v_{liquid}$ and $v_{gas}$ . Then the equilibrium state changes at the critical temperature $θ_{c} = 100 ° C$ ,

{\begin{matrix} W_{v_{liquid}} & 0 ° C < θ < 100 ° C W_{v_{gas}} & θ > 100 ° C \end{matrix} .

Importantly, while small variations in the temperature away from $θ_{c}$ will change the free energy of each state, it will not change the configuration of these minima with respect to the free energy. In other words, the system will still be a liquid for any $θ \in (0, 100)$ - its qualitative properties are stable. This is the content of a phase.

What is a phase?

A phase of a system is a region of configuration space $W \subset W$ that minimises the free energy, and is invariant to small perturbations in a relevant hyperparameter $θ$ . Typically, phases are distinguished by some macroscopic variable, in our case the volume $V (w)$ distinguishing subsets $W_{v}$ . More generally though, a phase describes some qualitative aggregate state of a system - like, as we've discussed in our example, the states of matter.

A phase is a path in the space of critical points — Phases are minima of the free energy that remain minima with small perturbations in $θ$ .

In some sense, you can define a phase to be any region that induces an equilibrium state with qualities you care about. But what makes phases a powerful concept is their relation to phase transitions - when there is a sudden jump in which state is preferred by the system.

What is a phase transition?

Phase transitions are changes in the structure of the global minima of the free energy, and often arise as non-analyticities of $F_{n}$ . This is a fancy way of saying they correspond to discontinuities in the free energy or one of its derivatives ^[4].

A first order phase transition at a critical temperature $θ_{c}$ corresponds to a reconfiguration of which phase is the global minima of the free energy.

First order phase transition exchanges minima — A first order phase transition exchanges global and local minima, changing the configuration of which phase is preferred according to which has the lowest free energy. These curves depict the phase transition as water reaches boiling point $θ = 100 ° C$ .

As we discussed above, heating water to boiling point $θ_{c} = 100 ° C$ is a classic example of a first order phase transition.

Two examples of second order phase transitions are where:

A merge transition occurs at $θ_{c}$ when two phases that are initially disjoint for $θ < θ_{c}$ merge to become the same state for $θ \geq θ_{c}$ , or;
A creation transition occurs at $θ_{c}$ when a local minima exists for $θ \geq θ_{c}$ but does not exist for $θ < θ_{c}$ . (If the directions are reversed, we call this a destruction transition).

(Note that we have not given a full classification of phase transitions here, because to do so one needs to study the possible types of catastrophes that can occur, as presented in [Gilmore]).

Phases in Statistical Learning

The notation and concepts in the previous section were not presented without reason. For starters, the Gibbs ensemble view of statistical learning is actually quite a rich analogy because, when the prior is uniform, the (random) Hamiltonian is equal to the empirical KL divergence ^[5],

H_{n} (w) = n K_{n} (w) .

The configuration space of microstates of the physical system then corresponds to parameter space $W$ with microstates given by different parameters $w \in W$ . This means the posterior is equivalent to the Gibbs probability distribution of the system being in a certain microstate, meaning the definition of free energy is identical. So, what exactly are the phases then?

In statistical learning then,

A phase corresponds to a local neighbourhood $W \subset W$ containing a singularity $w_{W}^{(0)}$ of interest.

To say that $W$ minimises the free energy is equivalent to saying that it has non-negligible posterior mass. The reason for this, as we explored in DSLT2, is that the singularity structure of a most singular optimal point $w_{W}^{(0)} \in W_{opt}$ dominates the behaviour of the free energy, because it minimises the loss $L (w)$ and has the smallest RLCT $λ_{W}$ .

You can, in principal, define a phase to be any region of $W$ . But the analysis of phases in the posterior only gets interesting when you have a set of phases that have fundamentally different geometric properties. The free energy formula tells us that these geometric properties correspond to different accuracy-complexity tradeoffs.

Consequently, in statistical learning, Watanabe states in [Wat18, $§ 9.4$ ] that

A phase transition is a drastic change in the geometry of the posterior as a function of a hyperparameter $θ$ .

Our definitions of first and second order phase transitions carry over perfectly from the physics discussion above.

It's important to clarify here that phase transitions in deep learning have many flavours. If one believes that SGD is effectively just "sampling from the posterior", then the conception that phase transitions are related to changes in the geometry of the posterior carries over. There is, however, one fundamentally different kind of "phase transition" that we cannot explain easily with SLT: a phase transition of SGD in time, i.e. the number gradient descent steps. The Bayesian framework of SLT does not really allow one to speak of time - the closest quantity is the number of datapoints $n$ , but these are not equivalent. We leave this gap as one of the fundamental open questions of relating SLT to current deep learning practice. ^[6]

The hyperparameter $θ$ can affect any number of objects involved in the posterior. Remembering that the posterior is

p (w | D_{n}) = \frac{φ (w) e^{- n L_{n} (w)}}{Z_{n}},

we could include hyperparameter $θ$ dependence in any of:

The model function $f (x, w) = f (x, w; θ)$ (i.e. the neural network defining $p (y | x, w)) .$
The true distribution $D_{n} = D_{n} (θ)$ , meaning $L_{n} (w) = L_{n} (w; θ)$ . (This could in principal be dependence in the input prior $q (x)$ or the actual dataset generated by $q (y | x)$ .)
The number of datapoints $n$ (inducing a first order phase transition due to the change in accuracy-complexity tradeoff).
The prior $φ (w) = φ (w; θ)$ .

Intuitive Examples to Interpret Phase Transitions

In DSLT2 we studied an example of a very simple one-dimensional $K (w)$ curve and got a feel for how the accuracy and complexity of a singularity affect the free energy of different neighbourhoods. Having now learned about phase transitions, we can cast new light on this example.

Example 1: First Order Phase Transition in $n$

Example 4.1: Consider again a KL divergence given by

K (w) = (w + 1)^{2} ((w - (1 + h_{C}))^{4} - k_{C})

where $w_{- 1}^{(0)} = - 1$ and $w_{1}^{(0)} = 1$ are the singularities, but the accuracy of $w_{1}^{(0)}$ is worse, $K (w_{1}^{(0)}) = C > 0$ . Then we can identify two phases corresponding to the two singularities,

W_{- 1} = B (w_{- 1}^{(0)}, δ) and W_{1} = B (w_{1}^{(0)}, δ)

for some radius $δ > 0$ such that the accuracy of $W_{- 1}$ is better, but the complexity of $W_{1}$ was smaller,

L (w_{- 1}^{(0)}) < L (w_{1}^{(0)}), but λ_{W_{- 1}} > λ_{W_{1}} .

As the hyperparameter $θ = n$ ^[7] varies, we see a first order phase transition at the critical value of $n_{c} \approx 17$ where the two free energy curves intersect, causing an exchange which phase is the global minima of the free energy. As we argued in that post, this is largely due to the accuracy-complexity tradeoff of the free energy. Notice also how the free energy of the global minima is non-differentiable at $n_{c}$ , showing an example of the "non-analyticity" of $F_{n}$ that we mentioned above.

Free energy crossover — A first order phase transition occurs at $n_{c}$ when the phase $W_{1}$ becomes the new global minima of the free energy.

Example 2: Second Order Merge Phase Transition

Example 4.2: We can modify our example slightly to observe a second order phase transition. Let's consider

K (w; θ) = (w + (1 - θ))^{2} (w - (1 - θ))^{4}

where $θ \in [0, 1]$ is a hyperparmeter that shifts the two singularities $w_{- 1}^{(0)} = - 1 + θ$ and $w_{1}^{(0)} = 1 - θ$ towards the origin. We will continue to label these phases $W_{- 1}$ and $W_{1}$ , noting their $θ$ dependence. ^[8]

Thus, at $θ_{c} = 1$ the two phases will merge and the KL divergence will be

K (w; 1) = w^{6} .

Therefore, at $θ = 1$ the singularity $w_{0}^{(0)} = 0$ will have an RLCT of

λ_{0} = \frac{1}{6} .

There is a new most singular point caused by the merging of two phases! Again, we can visually depict this phase transition:

2nd order phase transition — A second order phase transition as two phases merge into one with a lower RLCT.

Now that we have the basic intuitions of SLT and phase transitions down pat, let's apply these concepts to the case of two layer feedforward ReLU neural networks.

Phase Transitions in Two Layer ReLU Neural Networks

The main claim of this sequence is that Singular Learning Theory is a solid theoretical framework for understanding phases and phase transitions in neural networks. It's now time to make good on that promise and bring all of the pieces together to understand an actual example of phase transitions in neural networks. The full details of these experiments are explained in my thesis, [Carroll, $§ 5.2$ ], but I will briefly outline some points here for the interested reader. All notation and terminology is explained in detail in DSLT3, so use that section as a reference.

If you are uninterested, just skip to the next subsection to see the results.

Experimental Setup

We will consider a (model, truth) pair defined by the simple two layer feedforward ReLU neural network models we studied in DSLT3. Phase transitions will be induced by varying true distribution by a hyperparameter $θ$ , meaning $D_{n} = D_{n} (θ)$ . Since we have a full classification of $W_{0}$ from DSLT3, we understand the phases of the system, and therefore we want to study how their differing geometries affect the posterior. As we explained in that post, the scaling and permutation symmetries are generic (they occur for all parameters $w \in W$ ), but the node-degeneracy and orientation-reversing symmetries only occur under precise configurations of the truth. Thus, we are interested in studying the how the posterior changes as we vary the truth to induce these alternative true parameters - the phases of our setup.

The posterior sampling procedure uses an MCMC variant called HMC NUTS, which is brilliantly explained and interpreted here. Estimating precise nominal free energy values, and particularly those of the RLCT $λ$ , using sampling methods is currently very challenging (as explained in [Wei22]). So, for these experiments, our inference about phases and phase transitions will be based on visualising the posterior and observing the posterior concentrations of different phases. With this in mind, the posteriors below are averaged over four trials, 20,000 samples each, for each fixed true distribution defined by $θ$ . (Bayesian sampling is very computationally expensive, even in simple settings).

To isolate the phases we care about, we can use the fact that the scaling symmetry and permutation symmetries of our networks are generic. To this end we will normalise the weights by defining the effective weight ${^w}_{i} = | q_{i} | w_{i}$ ^[9], which preserves functional equivalence $f (x, w) = f (x,^w)$ ^[10]. We will say a node is degenerate if ${^w}_{i} = 0$ . We also project different node indices on to the same $({^w}_{i, 1}, {^w}_{i, 2})$ axes as follows:

Unnormalised vs normalised scatterplot of samples — Scatterplots of $20,000$ posterior samples $w^{(k)}$ for a two layer network: non-normalised weights $(w_{i, 1}, w_{i, 2})$ (left), and then normalised effective weights $({^w}_{i, 1}, {^w}_{i, 2})$ (right). For the given sample $w^{(k)}$ , each of the two nodes ${^w}_{1}, {^w}_{2}$ are projected on to the same plot (i.e. there are 40,000 dots on each of these plots).

The prior on inputs $q (x)$ is uniform on the square $[- 1, 1]^{2}$ , and the prior on parameters $φ (w)$ is the standard multidimensional normal $N (0, 1)$ .

Phase Transition 1 - Deforming to Degeneracy

In this experiment we will see a first order phase transition induced by deforming a true network from having no degenerate nodes to having one (possibility of a) degenerate node, as discussed in DSLT3 - Node Degeneracy. This example will reinforce the key messages of Watanabe's free energy formula: true parameters are preferred according to their RLCT, and at finite $n$ non-true parameters can be preferred due to the accuracy-complexity tradeoff.

Defining the Model, Truth, and Phases

We are going to consider a model network with $d = 2$ nodes,

f (x, w) = R e L U (⟨ {^w}_{1}, x ⟩ + {^b}_{1}) + R e L U (⟨ {^w}_{2}, x ⟩ + {^b}_{2}) + c

and a realisable true network $f (x, w^{(0)})$ with $m = 2$ nodes, which we will denote by $f_{2} (x, θ) := f (x, w^{(0)})$ to signify its hyperparameter $θ$ dependence (and distinguish it from the next experiment),

f_{2} (x, θ) = R e L U (⟨ {^w}_{1}^{(0)}, x ⟩ - \frac{1}{3}) + R e L U (⟨ {^w}_{2}^{(0)}, x ⟩ - \frac{1}{3}) .

The true weights rotate towards one another by a hyperparameter $θ = [0, \frac{π}{2}]$ , so ^[11]

w_{1}^{(0)} = (cos θ, sin θ), w_{2}^{(0)} = (- cos θ, sin θ) .

As we explained in DSLT3, we can depict the function and its activation boundaries pictorially:

PT1 Activation Boundaries — Contour plot of $f_{2} (x, θ)$ and its activation boundaries at $θ = 0$ (left), $θ = \frac{π}{4}$ (middle) and $θ = \frac{π}{2}$ (right).

At $θ = \frac{π}{2}$ , the truth could be expressed by a network with only one node, $m = 1$ ,

f_{2} (x, \frac{π}{2}) = R e L U (x_{2} - \frac{1}{3}) + R e L U (x_{2} - \frac{1}{3}) = 2 R e L U (x_{2} - \frac{1}{3}) .

This degeneracy is what we are interested in studying. The WBIC tells us to expect the posterior to prefer the one-degenerate-node configuration since it has less effective parameters. ^[12]

To identify our phases, at $θ = \frac{π}{2}$ there are two possible configurations of the effective model weights that are true parameters:

Both non-degenerate but share the same activation boundary: Both ${^w}_{1}, {^w}_{2} \neq 0$ such that ${^w}_{1} + {^w}_{2} = (0, 2)$ .
One degenerate, one non-degenerate: Either ${^w}_{1} = (0, 0)$ and ${^w}_{2} = (0, 2)$ , or vice versa by permutation symmetry.

To study these configurations we thus define phases based on annuli in the plane centred on the circle of radius $r$ with annuli radius of $ε$ ,

A(r,ε)={(^w

AI ALIGNMENT FORUM
AF

AI ALIGNMENT FORUM
AF

16

DSLT 4. Phase Transitions in Neural Networks

16

Phases Correspond to Singularities