Search papers, labs, and topics across Lattice.
This paper investigates the limitations of standard neural tangent kernel spectrum-aware pruning (NTK-SAP) in preserving the essential parameters of physics-informed neural networks (PINNs) for solving nonlinear partial differential equations (PDEs). The authors introduce a novel approach called physics-informed spectrum-aware pruning (PI-SAP), which prioritizes parameters based on their sensitivity to the PDE residuals, leading to improved preservation of solution fidelity. Experimental results demonstrate that PI-SAP consistently maintains residual fidelity across various equations while remaining competitive even under high sparsity, although optimal criteria vary by equation and sparsity level.
Foresight pruning can significantly enhance the performance of sparse PINN solvers by focusing on the sensitivity of PDE residuals rather than just output dynamics.
Physics-informed neural networks (PINNs) often rely on over-parameterized models to optimize coupled solution and differential-residual objectives, leaving unclear how much capacity is necessary and what pruning should preserve. We study foresight pruning at initialization for sparse PirateNet PDE solvers. Standard neural tangent kernel spectrum-aware pruning (NTK-SAP) aims to preserve output-side training dynamics but may overlook parameters whose main influence arises through derivatives in the governing equations. We introduce physics-informed spectrum-aware pruning (PI-SAP), which assigns saliency using sensitivity of the PDE residual. Experiments on the Gray-Scott equations, complex Ginzburg-Landau equation, Burgers'equation, and linear convection equation show that PI-SAP more consistently preserves Gray-Scott residual fidelity and is competitive under aggressive sparsity. However, no criterion is uniformly optimal across equations or sparsity levels. Small-batch PINN-NTK diagnostics further show that residual fidelity, solution accuracy, and kernel conditioning are distinct objectives, motivating pruning methods that explicitly balance solution-side and residual-side training dynamics during optimization.
