Search papers, labs, and topics across Lattice.
This paper introduces an adaptive wavelet-based physics-informed neural network (AW-PINN) to mitigate loss imbalances in PDEs with localized high-magnitude source terms. AW-PINN dynamically adjusts the wavelet basis function based on residual and supervised loss, avoiding reliance on automatic differentiation for derivative calculation, thus accelerating training. Experiments on challenging PDEs with extreme loss imbalances (up to $10^{10}:1$) demonstrate that AW-PINN consistently outperforms existing PINN methods.
PINNs get a wavelet makeover, adaptively focusing on high-magnitude source regions and leaving vanilla methods in the dust on PDEs with extreme loss imbalances.
In recent years, physics-informed neural networks (PINNs) have gained significant attention for solving differential equations, although they suffer from two fundamental limitations, namely, spectral bias inherent in neural networks and loss imbalance arising from multiscale phenomena. This paper proposes an adaptive wavelet-based PINN (AW-PINN) to address the extreme loss imbalance characteristic of problems with localized high-magnitude source terms. Such problems frequently arise in various physical applications, such as thermal processing, electro-magnetics, impact mechanics, and fluid dynamics involving localized forcing. The proposed framework dynamically adjusts the wavelet basis function based on residual and supervised loss. This adaptive nature makes AW-PINN handle problems with high-scale features effectively without being memory-intensive. Additionally, AW-PINN does not rely on automatic differentiation to obtain derivatives involved in the loss function, which accelerates the training process. The method operates in two stages, an initial short pre-training phase with fixed bases to select physically relevant wavelet families, followed by an adaptive refinement that adapts scales and translations without populating high-resolution bases across entire domains. Theoretically, we show that under certain assumptions, AW-PINN admits a Gaussian process limit and derive its associated NTK structure. We evaluate AW-PINN on several challenging PDEs featuring localized high-magnitude source terms with extreme loss imbalances having ratios up to $10^{10}:1$. Across these PDEs, including transient heat conduction, highly localized Poisson problems, oscillatory flow equations, and Maxwell equations with a point charge source, AW-PINN consistently outperforms existing methods in its class.
