Search papers, labs, and topics across Lattice.
This paper introduces a novel method for adapting neural network architecture by leveraging a posteriori error estimation, framing the training process as a continuous-time optimal control problem. By deriving explicit error bounds that quantify approximation error across network layers, the authors enable targeted insertion of new layers where the estimated error is highest, enhancing the network's ability to model complex, nonlinear phenomena. Experimental results on scientific datasets, including the Navier-Stokes equation, show that this approach significantly improves generalization performance compared to existing methods.
Targeted layer insertion based on rigorous error estimation leads to superior generalization in neural networks, outperforming traditional architecture adaptation techniques.
This work presents a novel approach for adapting neural network architecture along the depth based on a posteriori error estimation. By formulating neural network training as a continuous-time optimal control problem, we derive rigorous error estimates that quantify how approximation error distributes across network layers. This error decomposition enables a principled depth adaptation strategy: new layers are inserted at locations of maximum estimated error, allowing the network to efficiently capture complex, nonlinear variations in the underlying problem. Our framework introduces a novel network architecture that treats weights and biases as piecewise linear functions varying across layers, with the error estimator bounding the discrepancy between this discrete representation and the true continuous optimal control solution. The approach leverages dual weighted residual methodology from finite element analysis to derive computable upper bounds on the functional error. A key theoretical contribution is the derivation of explicit error bounds that decompose the total approximation error into interval-wise contributions, providing a rigorous basis for targeted architecture refinement. We demonstrate the effectiveness of our method on scientific datasets, including learning the observable-to-parameter map for the Navier-Stokes equation. Numerical results reveal that our approach consistently outperforms existing architecture adaptation methods in terms of generalization performance.
