Search papers, labs, and topics across Lattice.
This paper introduces a variational boosting framework for Physics-Informed Neural Networks (PINNs) that addresses issues of ill-conditioning and optimization instability by constructing solutions additively in function space. By training small correction networks that adhere to a local orthogonality condition, the method allows for stable second-order optimization techniques like Newton or conjugate gradient updates, which are typically challenging in large PINNs. The approach effectively transforms the global nonlinear refinement of solutions into a series of well-conditioned subproblems, enhancing the performance and stability of PINNs in solving nonlinear differential equations.
Variational boosting transforms PINNs from ill-conditioned monoliths into a series of stable, well-structured subproblems, enabling robust second-order optimization.
Physics-Informed Neural Networks (PINNs) solve differential equations by minimizing the residual of a nonlinear operator over a neural parameterization of the solution. However, monolithic PINNs often suffer from ill-conditioning, spectral bias, and optimization instability. We introduce a variational boosting framework in which solutions are constructed additively in function space. Each stage trains a weak learner whose converged correction satisfies a local orthogonality condition, equivalent to a projected functional gradient descent step onto the tangent space of the network's function manifold. Because each correction network is deliberately small, the restricted minimization admits full Newton or conjugate gradient updates, which are typically infeasible in large PINNs. The resulting method separates global nonlinear refinement into a sequence of well-conditioned subproblems while preserving the full variational structure of the operator. This framework provides a geometric interpretation of multi-stage PINNs as projected functional gradient descent and enables stable second-order optimization for nonlinear differential equations.
