Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces WaveLiT, a neural PDE solver architecture that leverages architectural inductive biases, specifically a discrete wavelet transform and multiscale feature pyramid, to achieve parameter efficiency. WaveLiT models with 1-10M parameters rival the performance of foundation models with 100-1000x more parameters on TheWell benchmarks. Analysis of a 10M-parameter WaveLiT variant reveals a physically interpretable transfer pattern, highlighting the importance of matching the architectural prior to the underlying dynamics.
Forget billion-parameter behemoths: a carefully crafted 10M-parameter architecture can outperform them at solving PDEs, revealing the power of inductive bias over brute force scaling.
Neural PDE solvers have followed the scaling trajectory of vision and language, with recent foundation models reaching billions of parameters. We argue that scale is a poor substitute for architectural inductive bias in this domain: structured priors deliver outsized parameter efficiency, and the pattern of where they succeed and fail is itself informative about what they capture. We instantiate this argument in WaveLiT, an architecture combining a discrete wavelet transform for lossless multi-resolution tokenization, an augmented linear attention block, a shared-weight multiscale feature pyramid, and a wavelet-domain auxiliary loss. Bespoke 1-10M-parameter WaveLiT models compete with foundation models of 100-1000$\times$ their size across eight TheWell benchmarks, with the largest gains on wave and acoustic-dominated benchmarks where the wavelet-multiscale prior fits the dominant dynamical structure and small per-step errors do not compound geometrically under rollout. Trained jointly across all eight benchmarks, a 10M-parameter foundation variant exhibits a structured, physically interpretable transfer pattern -- strongest where the wavelet-multiscale prior matches the dynamics, weakest on chaotic advection-dominated flows. The entire pipeline trains on a single GPU. The results suggest that small-model PDE performance is shaped by architectural inductive bias rather than scale, and that the structure of a prior's failures is a useful empirical signal about its content.