Search papers, labs, and topics across Lattice.
This paper introduces a Riemann-normal-coordinate Levenberg-Marquardt method (RNC-LM) that enhances the traditional LM method for nonlinear least-squares optimization by incorporating higher-order geometric updates. By reformulating the geodesic equation, RNC-LM provides finite-step updates that maintain consistency with the underlying manifold structure, significantly improving convergence and robustness in challenging optimization landscapes. Experimental results demonstrate that RNC-LM not only reduces errors in physics-informed neural networks but also achieves a remarkable 34-fold speedup in large-scale fitting tasks compared to the standard LM method.
RNC-LM achieves a 34-fold speedup in potential-energy-surface fitting while maintaining robust convergence in complex optimization scenarios.
Nonlinear least-squares optimization is central to regression, physics-informed neural networks, and other machine-learning tasks. Such problems have a natural geometric interpretation, model predictions form a manifold in data space, while the chosen parameterization can introduce parameter-effects curvature that becomes a dominant source of nonlinearity. This exposes a limitation of the Levenberg-Marquardt (LM) method, its tangent-space step is applied as a straight update in parameter coordinates. Geodesic acceleration gives a second-order correction, but its removal of parameter-effect curvature is exact only in the infinitesimal-step limit. We propose a Riemann-normal-coordinate Levenberg-Marquardt method (RNC-LM) to improve this consistency for finite optimization steps. By reformulating the geodesic equation, RNC-LM extends geodesic acceleration to arbitrary-order corrections and constructs finite-step updates with progressively higher reparameterization consistency. A line search along the resulting RNC curve controls the traveled distance while keeping the cost close to standard LM. The method eliminates the tangential component of residual acceleration order by order in a moving tangent frame, making the actual objective reduction more consistent with the linear model prediction of LM. On classical nonlinear least-squares benchmarks, RNC-LM improves convergence and robustness in curved valleys and rank-deficient problems. On a reaction-diffusion PINN failure-mode benchmark, it reduces the relative L2 error to the order of 1e-3 and recovers a physically meaningful solution. On a large-scale machine-learning potential-energy-surface fitting task, it achieves a 34-fold speedup over standard LM.
