Search papers, labs, and topics across Lattice.
This paper introduces a unifying "neural flow" framework encompassing both finite-dimensional neural networks and infinite-dimensional neural operators, based on continuous-depth models with composition and separation structures. It establishes well-posedness and proves universal approximation theorems for these neural flows, including the first such result for flow-based models between infinite-dimensional spaces. By discretizing the flow equations, the framework recovers both ResNet-type and plain architectures, providing a unified perspective on these common architectures.
Neural ODEs aren't just for continuous-time dynamics anymore: this work shows they can unify and universally approximate both finite and infinite-dimensional operators, bridging the gap between NNs and neural operators.
We introduce an abstract neural flow framework for neural networks and neural operators. The framework contains two continuous-depth models, namely neural flows with composition and separation structures, and covers both finite-dimensional function approximation and infinite-dimensional operator approximation. We prove well-posedness and universal approximation properties for the corresponding neural flows, including, to the best of our knowledge, the first universal approximation result for flow-based models between infinite-dimensional spaces. We also obtain universal approximation results for convolutional neural flow models. Through suitable time discretizations, the composition structure recovers ResNet-type architectures, while the separation structure, via a splitting-based discretization, yields plain architectures. This gives a unified flow-based route to both residual and plain architectures for neural networks and neural operators with fully connected or convolutional linear layers.
Tsinghua AI