Search papers, labs, and topics across Lattice.
The paper introduces continuous-time piecewise-linear recurrent neural networks (cPLRNNs) for dynamical systems reconstruction, addressing the limitations of discrete-time PLRNNs and the tractability issues of Neural ODEs. They develop a training and simulation algorithm that avoids numerical integration by leveraging the PL structure, enabling efficient computation and analysis. Experiments on dynamical systems reconstruction benchmarks, including systems with discontinuities, demonstrate the effectiveness of cPLRNNs compared to discrete-time PLRNNs and Neural ODEs, and show how topological objects can be determined semi-analytically.
Continuous-time piecewise-linear RNNs leapfrog discrete-time models and Neural ODEs in dynamical systems reconstruction by exploiting piecewise linearity for efficient training and analysis.
In dynamical systems reconstruction (DSR) we aim to recover the dynamical system (DS) underlying observed time series. Specifically, we aim to learn a generative surrogate model which approximates the underlying, data-generating DS, and recreates its long-term properties (`climate statistics'). In scientific and medical areas, in particular, these models need to be mechanistically tractable -- through their mathematical analysis we would like to obtain insight into the recovered system's workings. Piecewise-linear (PL), ReLU-based RNNs (PLRNNs) have a strong track-record in this regard, representing SOTA DSR models while allowing mathematical insight by virtue of their PL design. However, all current PLRNN variants are discrete-time maps. This is in disaccord with the assumed continuous-time nature of most physical and biological processes, and makes it hard to accommodate data arriving at irregular temporal intervals. Neural ODEs are one solution, but they do not reach the DSR performance of PLRNNs and often lack their tractability. Here we develop theory for continuous-time PLRNNs (cPLRNNs): We present a novel algorithm for training and simulating such models, bypassing numerical integration by efficiently exploiting their PL structure. We further demonstrate how important topological objects like equilibria or limit cycles can be determined semi-analytically in trained models. We compare cPLRNNs to both their discrete-time cousins as well as Neural ODEs on DSR benchmarks, including systems with discontinuities which come with hard thresholds.
