DSLT 2. Why Neural Networks obey Occam's Razor

Liam Carroll

Review

TLDR; This is the second main post of Distilling Singular Learning Theory which is introduced in DSLT0. I synthesise why Watanabe's free energy formula explains why neural networks have the capacity to generalise well, since different regions of the loss landscape have different accuracy-complexity tradeoffs. I also provide some simple intuitive examples that visually demonstrate why true parameters (i.e. optimally accurate parameters) are preferred according to the RLCT as , and why non-true parameters can still be preferred at finite $n$ if they have lower RLCT's, due to the accuracy-complexity tradeoff. (The RLCT is introduced and explained in DSLT1).

It is an amazing fact that deep neural networks seem to have an inductive bias towards "simple" models, suggesting that they obey a kind of Occam's Razor:

Plurality should not be posited without necessity.

or in modern parlance,

If two models of the world are similarly accurate, the simpler explanation should be preferred.

This allows them to achieve exceptionally low generalisation error despite classical statistics predictions that they should overfit data:

Double descent observed by OpenAI — Neural networks seem to obey a kind of double descent where bigger is better, breaking classical statistics predictions.
(Source: OpenAI's Double Descent blogpost).

This fact has come to be known as the generalisation problem and has been discussed at length in Zhang et. al 2017 (and a 2021 supplement), and in Bengio et al., amongst countless others.

Remarkably, Singular Learning Theory can help explain why neural networks, which are singular models, have the capacity to generalise so well.

The degeneracy of the Fisher information matrix is actually a feature of singular models, not a bug. This is because different regions of parameter space can have different complexities as measured by the RLCT $λ$ , unlike regular models where the complexity is fixed to the total number of parameters in the model $d$ . This is the implicit content of Watanabe's profound free energy formula, called the Widely Applicable Bayesian Information Criterion (WBIC), which quantifies a precise asymptotic tradeoff between inaccuracy and complexity,

W B I C = n L_{n} (w^{(0)})      inaccuracy + λ    complexity log n,

giving a mathematically rigorous realisation of Occam's Razor, since $λ \leq \frac{d}{2}$ in singular models.

In this post we will explore Watanabe's free energy formula and provide an intuitive example of why the RLCT matters so much. If you are new to statistical learning theory, I would recommend jumping straight to the examples and their related animations to gain the intuition first, and then return to the theory afterwards.

The four key points to take away are:

As $n \to \infty$ , true parameters with the best accuracy will always be preferred.
As $n \to \infty$ , if two true parameters are equally accurate but have different RLCT's, the parameter with the lower RLCT is preferred.
For finite but large $n$ , non-true parameters can be preferred by the posterior because of an accuracy-complexity tradeoff as measured by the WBIC.
Parameters with low inaccuracy and small RLCT's $λ$ have low generalisation error (in a Bayesian sense) since the Bayes generalisation error $G_{n}$ is the "derivative" of the free energy, so

G_{n} = L_{n} (w^{(0)}) + \frac{λ}{n} + o (\frac{1}{n}) .

Information Criteria Help Avoid Underfitting and Overfitting

In the last post, we derived the asymptotic free energy $F_{n}$ as $n \to \infty$ for regular models, called the Bayesian Information Criterion (BIC):

B I C = n L_{n} (w^{(0)}) + \frac{d}{2} log n,

where $n$ is the total number of datapoints in the dataset $D_{n}$ , the optimal loss is $L_{n} (w^{(0)})$ where $w^{(0)} \in W$ is a maximum likelihood estimate (i.e. $w^{(0)} = {a r g m i n}_{W} (L (w))$ ), and $d$ is the total dimension of parameter space $W \subseteq R^{d}$ .

As a statistical practitioner, given some dataset $D_{n}$ , your goal is to find a model that you hope will represent the truth from some candidate list. You only have access to the truth via your (training) dataset $D_{n}$ , but you also want to ensure that it generalises to data beyond the dataset. You can use the BIC to compare model candidates across a set of model classes that you can think to compare, since it captures a precise asymptotic tradeoff between inaccuracy $L_{n} (w^{(0)})$ and complexity $\frac{d}{2}$ . Under this paradigm, we should choose the model that achieves the lowest BIC as it is the best option for avoiding both underfitting and overfitting the data. Let's consider a simple example in action:

Example 1: Suppose we have $n = 61$ datapoints drawn from a quadratic with Gaussian noise, $y = x^{2} + ε$ where $ε \sim N (0, {0.15}^{2})$ ^[1], where $x$ is drawn according to a uniform prior $q (x) = \frac{1}{2} 1 (x \in [- 1, 1])$ . After looking at our scatterplot of data $D_{n}$ , we could try models across the following set of model classes:

Name	$d$	Model
Linear	2	$f_{1} (x, w) = w_{1} x + w_{0}$
Quadratic	3	$f_{2} (x, w) = w_{2} x^{2} + w_{1} x + w_{0}$
Cubic	4	$f_{3} (x, w) = w_{3} x^{3} + w_{2} x^{2} + w_{1} x + w_{0}$
Degree 15	16	$f_{15} (x, w) = w_{15} x^{15} + w_{14} x^{14} + \dots + w_{1} x + w_{0}$

(The degree 15 model is an extremity just to illustrate a point.)

Within each model class, we can then perform ordinary least squares regression ^[2] to find the model fit $w^{(0)}$ with optimal loss $L_{n} (w^{(0)})$ (which, in the regression case, is simply the mean-squared-error plus a constant^[3]). With the optimal loss of each model class in hand, we can then compare the $B I C$ over our set of candidates.

A scatterplot for quadratic data — The optimal model fit to $D_{n}$ for each model class, which vary in $B I C$ value.

Model	$L_{n} (w^{(0)})$	$d$	$B I C$
$f_{1} (x, w)$	0.9668	1	63.09
$f_{2} (x, w)$	0.9283	2	62.80
$f_{3} (x, w)$	0.9280	3	64.83
$f_{15} (x, w)$	0.9243	15	87.22

As one expects, in a similar vein to the bias-variance tradeoff, there is a clear optimal (lowest) $B I C$ . As the dimension increases, the accuracy gets better and better, but at the cost of the complexity of the model (and therefore, its generalisability^[4]). The linear model is simple, but has high loss. The cubic has marginally lower loss, but at the expense of a complexity increase that isn't worth it. The degree 15 polynomial has the lowest loss of them all, but is penalised heavily for its complexity (as it should be - it is clearly overfitting). The Goldilocks choice is unsurprisingly the quadratic model, $f_{2} (x)$ , because the tradeoff between accuracy and complexity is just right.

Other than the fact that the $B I C$ simply does not hold in the singular case, it also points us towards a limitation of regular models. Once you pick your model class to optimise, every point on the loss landscape has a fixed model complexity. If your goal is to minimise the $B I C$ , you only have one choice: to find the single point that optimises the loss at the bottom of the well.

In fact, in our particular case, we can calculate the KL divergence ^[5] for the linear model $f_{2} (x, w)$ ,

K (w_{0}, w_{1}) = \frac{1}{6} w_{1}^{2} + \frac{1}{2} {(w_{0} - \frac{1}{3})}_{0}^{2} + \frac{2}{45},

which we can see a plot of below.

Regular K(w) — For a regular linear model with $d = 2$ , parameters $K (w)$ is just a paraboloid. Every point on its surface has the same complexity, $λ = \frac{d}{2} = 1$ , measuring the effective dimensionality.

In singular models, this all changes: within the same model class, different models $w$ in parameter space $W$ have different effective dimensionalities as measured by the RLCT $λ$ . The learning procedure does the work of the statistician for us, because the loss landscape contains information about both the accuracy and the complexity.

Watanabe's Free Energy Formula for Singular Models

Free Energy, Generalisation and Model Selection

Fundamentally, we care about the free energy $F_{n} = - log Z_{n}$ because it is a measure of posterior concentration, and as we showed with the BIC calculation in DSLT1, it tells us something about the information geometry of the posterior. In particular, given a compact neighbourhood of parameter space $W \subseteq W$ we can define its local free energy

F_{n} (W) = - log (\int_{W} φ (w) e^{- n L_{n} (w)} d w),

providing a direct tool for comparing different regions of the posterior, and thus different models. Since there is a correspondence

Small F_{n} (W) ⟺ Large posterior concentration \int_{W} p (w | D_{n}) d w,

we say the posterior prefers a region $W$ when it has low free energy relative to other regions of $W$ .

But this isn't the only reason to care about the free energy. In fact, it is explicitly related to generalisation, at least in the Bayesian sense.

In the frequentist framework that real-world deep learning takes place in (i.e. estimating a single parameter $^w$ using SGD), we typically split our dataset $D_{n}$ into a training set and a test set. We then say a model defined by $^w$ generalises well when it has low loss on the test set - in other words, it performs well on data that it hasn't seen before.

In the Bayesian paradigm, generalisation can be formulated according to a number of different estimation methods and quantities, depending on how you extract information from the posterior (e.g. you could estimate a maximum-likelihood parameter, a maximum a posterior estimate, or an average over samples from the posterior, etc.). Let's focus on one for the moment that involves the Bayes predictive distribution given by

p (y | x, D_{n}) = E_{w} [p (y | x, w)] = \int_{W} p (y | x, w) p (w | D_{n}) d w,

which weights the probability of an output $y$ given an input $x$ according to the posterior measure. The Bayes generalisation loss is then given by

G_{n} = E_{X} [- log p (y | x, D_{n})] = - \iint_{R^{N + M}} q (y, x) log p (y | x, D_{n}) d x d y .

Intuitively, it is the expected loss of the predictive distribution over all possible inputs $x$ and outputs $y$ . It can be shown with relative ease ^[6] that the Bayes generalisation loss is equal to the average increase in free energy,

G_{n} = E_{X_{n + 1}} [F_{n + 1}] - F_{n} .

In an informal-yet-conceptually-correct way, we can treat this difference as being the "derivative with respect to $n$ " ^[7].

It follows immediately that the generalisation loss of a region $W \subseteq W$ is

G_{n} (W) = E_{X_{n + 1}} [F_{n + 1} (W)] - F_{n} (W) .

Ergo, to understand the information contained in the posterior, and which regions contain models with low generalisation error, we want to calculate the free energy. In even modestly simple settings this integral is intractable, thus why we need to calculate its asymptotic form.

The Free Energy Formula for Singular Models

This subsection is a little bit more technical. If this overwhelms you, I recommend skipping ahead to the next subsection where I interpret the free energy formula.

As I explained in DSLT1, finding the asymptotic form of the free energy $F_{n}$ as $n \to \infty$ when $I (w)$ is degenerate is hard, and depends of theorems of algebraic geometry and distribution theory. This formula has been refined over the course of many papers ^[8], adjusted and generalised with various hypotheses. Here we will focus on the form given in [Wat13], which also applies to the unrealisable case where the set of true parameters $W_{0}$ may be empty. Thus we instead care about the set of optimal parameters

W_{opt} = {w \in W | L (w) = min w^{'} \in W L (w^{'})},

and if $W_{0}$ is non-empty, then $W_{0} = W_{opt}$ . As it stands, the free energy formula in the unrealisable case depends on a hypothesis of relatively finite variance ^[9].

Watanabe shows that the free energy of $W$ asymptotically satisfies

F_{n} = n L_{n} (w^{(0)}) + λ log n + U_{n} \sqrt{\frac{λ log n}{2}} + O_{p} (1) as n \to \infty .

where:

$n$ is the size of the dataset $D_{n}$ .
$w^{(0)} \in W_{opt}$ is a most singular optimal point of $W$ (which we will explain below), which is in the interior of $W$ .
$λ \in Q_{> 0}$ is the global RLCT, associated to the most singular point $w^{(0)}$ .
${U_{n}}$ is a sequence of random variables that satisfies $E [U_{n}] = 0$ and converges to a Gaussian random variable in law as $n \to \infty$ .
$O_{p} (1)$ denotes a sequence that is bounded in probability.

We will interpret the formula in a moment, but let me briefly clarify what it means to call $w^{(0)}$ a "most singular point", a notion is made precise in [Lin11, Proposition 3.9]. (See the below figure, too).

The gist is that every singularity $w \in W_{opt}$ has an associated local RLCT $λ_{w}$ defined by considering small neighbourhoods around the point. The global RLCT of $W$ is defined to be the minimum over these optimal points, $λ = {min}_{w \in W_{opt}} {λ_{w}}$ , and an optimal point $w^{(0)} \in W_{opt}$ is a most singular point if $λ_{w^{(0)}} = λ$ ^[10]. Note also that the formula currently depends on an optimal parameter being in the interior of $W$ , so you can think of a most singular point as being a local minimum of $K (w)$ .

The Widely Applicable Bayesian Information Criterion

In the asymptotic limit as $n \to \infty$ , we can ignore the last two terms of the free energy formula and arrive at the Widely Applicable Bayesian Information Criterion (WBIC) across the full parameter space $W$ ,

W B I C = n L_{n} (w^{(0)}) + λ log n .

Notice how the WBIC formula is the same as the BIC except that complexity is measured by $λ$ instead of $\frac{d}{2}$ . But in DSLT1, we explained how

In regular models $λ = \frac{d}{2}$ .
In singular models $λ \leq \frac{d}{2}$ in general.

Thus the WBIC is a generalisation of the BIC, containing the regular result as a special case.

Though the WBIC can be used to compare model classes like in the case of the BIC, its real power is what it tells us about the information geometry of the posterior within the same class of models. To this end, we can calculate the local free energy of a compact neighbourhood $W \subseteq W$

F_{n} (W) = n L_{n} (w_{W}^{(0)}) + λ_{W} log n,

where $w_{W}^{(0)} \in W_{opt}$ is the most singular optimal point in $W$ with associated RLCT $λ_{W}$ .

Most singular point — Here $w_{1}^{(0)}$ is the most singular point of the local neighbourhood $W_{1}$ , and $w_{2}^{(0)}$ is the most singular point of $W_{2}$ . Globally, $w_{1}^{(0)}$ is the most singular point of all true parameters in $W_{0}$ which means it dominates the total free energy $F_{n} .$ In particular, $F_{n} (W_{1}) < F_{n} (W_{2})$ since $λ_{1} < λ_{2}$ .
(Thanks to Jesse for the figure template).

The Accuracy-Complexity Tradeoff

The WBIC shows that the free energy of a region $W \subseteq W$ is comprised of the accuracy and complexity (or, to the physicists, energy and entropy) of the most singular point in $W$

F_{n} (W) = n L_{n} (w_{W}^{(0)})      inaccuracy + λ_{W}    complexity log n .

What makes this so profound is that in singular models, different regions $W \subset W$ can have different RLCT's $λ_{W}$ , each with a different tradeoff between accuracy and complexity, unlike the regular model case where every region has a fixed complexity:

\begin{matrix} Regular models & : F_{n} (W) = n (Inaccuracy (W)) + log n (Constant Complexity) Singular models & : F_{n} (W) = n (Inaccuracy (W)) + log n (Complexity (W)) \end{matrix}

So, the region in $W$ that minimises the free energy has the best accuracy-complexity tradeoff. This is the sense in which singular models obey Occam's Razor: if two regions are equally accurate, then they are preferred according to which is the simpler model.

Interpreting the terms in the free energy formula leads us to three main points:

As $n \to \infty$ , regions $W$ containing optimal parameters $w^{(0)} \in W_{opt}$ will always be preferred, since the inaccuracy is the leading order term.
As $n \to \infty$ , if there are multiple regions $W_{1}$ and $W_{2}$ with equally optimal accuracy, then they are preferred according to their respective RLCT's. Lower is better, so if $λ_{1} < λ_{2}$ then $W_{1}$ is preferred by the posterior.
For finite but large $n$ , regions that do not contain a globally optimal parameter can be preferred by the posterior because they can have a better tradeoff between accuracy and complexity.

Why Singular Models (Can) Generalise Well

Armed with our free energy formula, we can now understand why singular models have the capacity to generalise well.

Recall that the Bayes generalisation error can be expressed as the difference $G_{n} = E_{X_{n + 1}} [F_{n + 1}] - F_{n}$ , which can be interpreted as the "derivative" of the free energy with respect to $n$ . Then since $F_{n} = n L_{n} (w^{(0)}) + λ log n$ , Watanabe is able to prove in [Wat18, Chapter 8] that, asymptotically, the Bayes generalisation error is

E [G_{n}] = L (w^{(0)}) + \frac{λ}{n} + o (\frac{1}{n}) .

In fact, he goes one step further by considering two other forms of generalisation, the leave-one-out-cross-validation-loss $C V$ , and the WAIC (an empirically measurable form of the WBIC), and shows that asymptotically

E [G_{n}] \approx E [CV] \approx E [WAIC] = L (w^{(0)}) + \frac{λ}{n} + o (\frac{1}{n}) .

Once again, we find that the RLCT $λ$ plays a central role in the learning process. Most importantly, we have a correspondence:

Optimal parameters w^{(0)} with low RLCT λ ⟺ Low generalisation error .

On top of this, we can carry out the same analysis on our local $F_{n} (W)$ to find that the local generalisation loss of $W$ is

E [G_{n} (W)] = L (w_{W}^{(0)}) + \frac{λ_{W}}{n} .

Since the RLCT can differ region to region, this tells us that:

In singular models, different regions W have different generalisation error.

All of this is to say: under any reasonable conception of Bayesian generalisation, the RLCT plays a central role in minimising generalisation loss, asymptotically. And since $λ \leq \frac{d}{2}$ in singular models, the generalisation error of a singular model will always be better than that of a regular model with the same number of parameters. In DSLT3 we will show neural networks are singular, so:

This is why neural networks can generalise well!

...sort of. Don't forget, we are in the Bayesian paradigm here, and it is not a given that Stochastic Gradient Descent (SGD) finds the same regions of parameter space that the Bayesian posterior says are "good". We postulate that in some sense they are equivalent, and the work of Mingard et al. in Is SGD a Bayesian Sampler? Well, Almost. agrees with this postulate. But, formalising this relationship does remain a key open problem in conclusively applying SLT to modern deep learning with SGD and its variants.

From points to local neighbourhoods

For those with less background knowledge, there is an important conceptual shift we have made here that I want to elaborate on briefly.

In frequentist statistics we care about particular point estimates $^w$ in parameter space $W$ . But in Bayesian statistics, we care about measurable regions $W \subset W$ of parameter space, and the probability to which the posterior assigns those regions. This is a powerful shift in perspective, and points towards why SLT is placed in a Bayesian paradigm: the observation the geometry of $K (w)$ contains a lot more information than simple point estimates do lends itself naturally to Bayesian statistics.

But in modern deep learning, we only ever have access to a point estimate $^w$ at the end of training via SGD. Sampling from the Bayesian posterior for large neural networks would not only be silly, but it would be, essentially, computationally impossible. So does this mean SLT has absolutely no applicability to modern deep learning?

Not at all. By studying the local geometry of the loss landscape $K (w)$ - which is to say, arbitrarily small neighbourhoods $W$ of the posterior - we are able to analyse the set of points that are arbitrarily close to the singularities of $K (w)$ .

What Watanabe shows is that the singularities contained in these small neighbourhoods affect the geometry of the other points in the neighbourhood. If $W$ contains one true parameter $w^{(0)} \in W_{0}$ , the other points in $W$ may not be equal minima of $K (w)$ , but they are extremely close to being so. The same logic applies for the RLCT of a region: perhaps there is only one most singular point in $W$ , but any nearby parameter within the small neighbourhood will define a model whose functional output is nearly identical to that of $f (x, w^{(0)})$ which has a lower complexity.

In focusing only on points, we lose an extraordinary amount of information. So, we localise to neighbourhoods.

Intuitive Examples to Interpret the WBIC

It's time we looked at a specific example to build intuition about what the WBIC is telling us.

I have constructed this toy example specifically to illustrate the main points here, and a lot of the details about the sub-leading terms in the free energy expansion are obfuscated, as well as the random fluctuations that make $K_{n} (w)$ different to $K (w)$ . But, it is conceptually correct, and helps to illustrate the dominant features of the learning process as $n \to \infty$ . Don't take it too literally, but do take it seriously.

We will start with some calculations, and then visualise what this means for the posterior as $n \to \infty$ .

Example 1: True parameters are preferred according to their RLCT

Let's consider a one parameter model $d = 1$ with KL divergence defined by

K (w) = (w + 1)^{2} (w - 1)^{4},

on the region $W = [- 2, 2]$ with uniform prior $φ (w) = \frac{1}{4} 1 (w \in W)$ . There are two singularities in the set of true parameters,

W_{0} = {- 1, 1},

which we will label as $w_{- 1}^{(0)}$ and $w_{1}^{(0)}$ respectively. The Fisher information at true parameters is just the Hessian, which in the one dimensional case is simply the second derivative,

\begin{matrix} I (w^{(0)}) = J (w^{(0)}) = \frac{d^{2} K}{d w^{2}} {∣ ∣}_{w = w^{(0)}} . \end{matrix}

An easy calculation shows

\frac{d^{2} K}{d w^{2}} = 2 (w - 1)^{2} (15 w^{2} + 10 w - 1),

meaning $w_{- 1}^{(0)}$ is a regular point and $w_{1}^{(0)}$ is a singular point since

d