Search papers, labs, and topics across Lattice.
This paper derives PAC generalization bounds for neural oscillators, specifically those consisting of a second-order ODE followed by an MLP, when approximating causal operators and stable dynamical systems. Using the Rademacher complexity framework, the authors show that the estimation error grows polynomially with the MLP size and time length, avoiding the curse of dimensionality. They also demonstrate theoretically and empirically that constraining the Lipschitz constants of the MLPs through regularization improves generalization performance, validated on a Bouc-Wen nonlinear system.
Regularizing Lipschitz constants in MLPs within neural oscillators provably and practically enhances generalization, offering a path to more robust learning of complex dynamical systems.
Neural oscillators that originate from the second-order ordinary differential equations (ODEs) have shown competitive performance in learning mappings between dynamic loads and responses of complex nonlinear structural systems. Despite this empirical success, theoretically quantifying the generalization capacities of their neural network architectures remains undeveloped. In this study, the neural oscillator consisting of a second-order ODE followed by a multilayer perceptron (MLP) is considered. Its upper probably approximately correct (PAC) generalization bound for approximating causal and uniformly continuous operators between continuous temporal function spaces and that for approximating the uniformly asymptotically incrementally stable second-order dynamical systems are derived by leveraging the Rademacher complexity framework. The theoretical results show that the estimation errors grow polynomially with respect to both the MLP size and the time length, thereby avoiding the curse of parametric complexity. Furthermore, the derived error bounds demonstrate that constraining the Lipschitz constants of the MLPs via loss function regularization can improve the generalization ability of the neural oscillator. A numerical study considering a Bouc-Wen nonlinear system under stochastic seismic excitation validates the theoretically predicted power laws of the estimation errors with respect to the sample size and time length, and confirms the effectiveness of constraining MLPs' matrix and vector norms in enhancing the performance of the neural oscillator under limited training data.
