Distilling Singular Learning Theory

This sequence distills Sumio Watanabe's Singular Learning Theory (SLT) by explaining the essence of its main theorem - Watanabe's Free Energy Formula for Singular Models - and illustrating its implications with intuition-building examples. I show why neural networks are singular models, and demonstrate how SLT provides a framework for understanding phases and phase transitions in neural networks.