Search papers, labs, and topics across Lattice.
This paper explores the statistical properties of deep learning through a physics-informed lens, highlighting the surprising deviations from classical statistical intuitions that enable deep learning to excel in real-world applications. By examining neural scaling laws and their relationship with inductive biases and constraints in physics-related machine learning tasks, the authors provide insights into model construction choices that impact performance. The findings underscore the complexity of deep learning models and the need for a nuanced understanding of their statistical foundations to optimize their application in scientific domains.
Neural scaling laws reveal unexpected insights into how deep learning models defy classical statistics, reshaping our understanding of model performance in scientific applications.
Deep learning has managed to evade numerous intuitions from classical statistics to achieve unprecedented performance on a number of real-world tasks. In this article, we investigate the key features and surprises of deep learning from a physics-informed perspective, taking care to point out and justify where possible the many choices inherent in constructing a deep learning model. In particular, we review the phenomenon of neural scaling laws and discuss their interplay with the constraints and inductive biases which may be present when applying machine learning to problems in physics.
