Search papers, labs, and topics across Lattice.
This paper extends the universal approximation theorem for functional input neural networks (FNN) to encompass differentiable maps by incorporating the approximation of derivatives. The authors establish a weighted Nachbin theorem that enables the universal approximation of differentiable maps on infinite-dimensional manifolds, surpassing traditional compact set limitations. Key results include the ability to approximate non-anticipative functionals and path space functionals, demonstrating that linear functions of the signature can effectively approximate their directional derivatives.
Universal approximation for differentiable maps now includes derivatives, unlocking new possibilities for function approximation in infinite-dimensional spaces.
We generalize the universal approximation theorem for functional input neural networks (FNN) to differentiable maps by including the approximation of the derivatives. A FNN maps the input from a possibly infinite-dimensional weighted manifold to the real-valued hidden layer, on which a non-linear scalar activation function is applied, and then returns the output into a Banach space via some linear readouts. By proving a weighted Nachbin theorem, we establish a universal approximation theorem (UAT) for differentiable maps, which goes beyond the usual formulation on compact sets and also includes the approximation of the derivatives. This leads us to approximation results for non-anticipative functionals including the horizontal and vertical derivatives. As a further application, we show that linear functions of the signature are able to approximate path space functionals including their directional derivatives.
