Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
We analyze why neural PDE solvers can fail to reach their best approximation due to unbounded parameter growth, identifying this as a consequence of either unbounded hidden parameters or redundant neurons. Our theoretical framework connects this phenomenon to missing limits in deep neural tanh-networks, specifically for models with translated kernels. By augmenting these models with kernel derivatives, we recover the missing functions and enable attainment of the best approximation, a finding validated by numerical studies on parameter growth and optimization.
Neural PDE solvers can miss optimal solutions due to unbounded parameters, but adding kernel derivatives recovers these "missing limits."
Neural solvers for partial differential equations (PDEs) can approach an accurate solution while their parameters grow without bound. In such cases, the limiting solution may have no finite representation in the chosen model, leaving the best loss unattained. Our analysis connects missing limits in deep neural tanh- networks to unbounded hidden parameters or increasingly redundant neurons. For a class of models built from translated kernels, we describe the missing functions and recover them by adding kernel derivatives to the model. This completion makes the best approximation attainable under standard assumptions. Numerical studies follow the associated parameter growth and explore how completion affects PDE optimization.