The Shallow Reality of 'Deep Learning Theory'

Classical learning theory makes the wrong assumptions, takes the wrong limits, uses the wrong metrics, and aims for the wrong objectives.

In this sequence, I review the current state of learning theory and the many ways in which it is broken. I argue that the field as it currently stands is in need of fix-up, and that developing a useful theory of deep learning will require turning elsewhere, likely to something that builds on singular learning theory.