Search papers, labs, and topics across Lattice.
This paper introduces and benchmarks curvature-aware optimization techniques, including Natural Gradient and quasi-Newton methods (Self-Scaling BFGS and Broyden), to accelerate and improve the convergence of Physics-Informed Neural Networks (PINNs) when solving challenging PDEs and ODEs. The authors also propose new PINN formulations for the inviscid Burgers and Euler equations. Results show that these advanced optimizers significantly outperform standard methods, enabling PINNs to achieve higher accuracy and faster convergence on complex physical systems, while also addressing the scalability challenges of quasi-Newton methods for batched training.
PINNs can now solve complex physics problems with unprecedented accuracy and speed, thanks to curvature-aware optimization techniques that blow away standard methods.
Efficient and robust optimization is essential for neural networks, enabling scientific machine learning models to converge rapidly to very high accuracy -- faithfully capturing complex physical behavior governed by differential equations. In this work, we present advanced optimization strategies to accelerate the convergence of physics-informed neural networks (PINNs) for challenging partial (PDEs) and ordinary differential equations (ODEs). Specifically, we provide efficient implementations of the Natural Gradient (NG) optimizer, Self-Scaling BFGS and Broyden optimizers, and demonstrate their performance on problems including the Helmholtz equation, Stokes flow, inviscid Burgers equation, Euler equations for high-speed flows, and stiff ODEs arising in pharmacokinetics and pharmacodynamics. Beyond optimizer development, we also propose new PINN-based methods for solving the inviscid Burgers and Euler equations, and compare the resulting solutions against high-order numerical methods to provide a rigorous and fair assessment. Finally, we address the challenge of scaling these quasi-Newton optimizers for batched training, enabling efficient and scalable solutions for large data-driven problems.
