 |
Generally IntelligentAuthor: Kanjun Qiu
Conversations with builders and thinkers on AI's technical and societal futures. Made by Imbue. Language: en Genres: Technology Contact email: Get it Feed URL: Get it iTunes ID: Get it |
Listen Now...
There will be a scientific theory of deep learning
Friday, 24 April, 2026
Deep learning works extraordinarily well. And we still largely don't know why.A new paper from Jamie Simon, Daniel Kunin, and 12 co-authors argues that a scientific theory of deep learning is emerging, and coins a name for the emerging field: learning mechanics.We sat down with Jamie and Dan on Generally Intelligent to talk about what a physics of deep learning would actually look like, why now, and what's left to figure out.00:03:05 Learning mechanics as the physics to mechanistic interpretability's biology00:04:13 Why deep learning needs a theory00:07:07 Why deep learning is uniquely hard to engineer00:12:11 How a week in the woods became a paper00:25:59 The barrier to theory isn't opacity, but complexity00:36:26 Deep learning's first gas law00:47:22 Why more particles makes the problem easier 00:56:22 The discretization hypothesis01:01:50 The strongest signal that a compact theory exists01:05:07 The Platonic Representation Hypothesis01:15:41 Why learning mechanics and mech interp need each other01:25:29 Theory as safety infrastructureRead the paperTranscript and linksLearning Mechanics website
