Search papers, labs, and topics across Lattice.
This paper introduces a convergent training scheme for neural networks utilizing analytic activation functions based on gradient flows, ensuring convergence through Lojasiewicz theory. The method simplifies the implementation of neural networks by approximating their coefficients via ordinary differential equations. Experimental results demonstrate the effectiveness of this approach in accurately approximating solutions to parameter-dependent ordinary differential equations and inverse problems with wave constraints, even in challenging ill-posed regions.
Achieving reliable approximations of solutions to ill-posed inverse problems using a straightforward neural network training scheme could revolutionize how we tackle complex parameter-dependent challenges.
We develop a convergent scheme to train neural networks involving analytic activation functions based on gradient flows. Convergence properties are guaranteed by Lojasiewicz theory. The main advantage of this approach is its simplicity of implementation. The coefficients of the network are approximated by solving a system of ordinary differential equations. We test the method by constructing residual neural network approximations of solutions of parametric problems. The dependence of the solutions of simple ordinary differential equations on a few parameters is correctly reproduced. The solutions of inverse problems involving wave constraints which depend on a few parameters can be reasonably approximated, even in regions in which the problem is severely ill posed.
