Search papers, labs, and topics across Lattice.
This paper addresses the fragility of training physics-informed neural networks (PINNs) by identifying a failure mode linked to model capacity, where overparameterized networks self-partition into exclusive modules that hinder effective learning. To combat this issue, the authors introduce Modular-Sparsity Synchronization (ModSync), a framework that penalizes task-exclusive connections while promoting cross-objective interactions. Experimental results across various PDE benchmarks show that ModSync not only prevents capacity-induced failures but also achieves state-of-the-art accuracy in solving PDEs.
Overparameterized PINNs can self-partition into ineffective modules, but a new training framework restores robust learning and achieves unprecedented accuracy.
Physics-informed neural networks (PINNs) have become a powerful framework for solving PDEs by embedding physical laws into differentiable objectives. Despite their advances, training PINNs remains fragile: recent conflict-averse optimization schemes alleviate gradient interference between residual and boundary losses, but we show that their effectiveness deteriorates as model capacity increases. In this paper, we identify a capacity-induced failure mode, where overparameterized networks undergo functional modularity, self-partitioning into task-exclusive modules that suppress cross-objective interaction and hinder convergence toward Pareto-stationary points. To address this issue, we propose a novel framework, Modular-Sparsity Synchronization (ModSync), which integrates structural optimization into conflict-averse training by penalizing task-exclusive connections while preserving interaction-promoting pathways. Extensive experiments across diverse PDE benchmarks demonstrate that ModSync consistently prevents capacity-driven failures, sustains robust cross-objective coupling, and achieves state-of-the-art accuracy. Codes are available at \url{https://github.com/heejokong/ModSync}.
