Search papers, labs, and topics across Lattice.
This paper introduces pseudo-differential enhanced physics-informed neural networks (pd-PINNs), which augment the PINN loss function with higher-order PDE residuals computed in Fourier space to improve training. By differentiating in Fourier space via multiplication with the Fourier wavenumber, pd-PINNs achieve faster convergence and lower error compared to standard PINNs, especially in low collocation settings. The method improves the spectral eigenvalue decay of the neural tangent kernel (NTK), facilitating the learning of high frequencies and mitigating frequency bias.
Differentiating PINN residuals in Fourier space accelerates training, improves accuracy, and overcomes frequency bias, particularly in scenarios with limited collocation points.
We present pseudo-differential enhanced physics-informed neural networks (PINNs), an extension of gradient enhancement but in Fourier space. Gradient enhancement of PINNs dictates that the PDE residual is taken to a higher differential order than prescribed by the PDE, added to the objective as an augmented term in order to improve training and overall learning fidelity. We propose the same procedure after application via Fourier transforms, since differentiating in Fourier space is multiplication with the Fourier wavenumber under suitable decay. Our methods are fast and efficient. Our methods oftentimes achieve superior PINN versus numerical error in fewer training iterations, potentially pair well with few samples in collocation, and can on occasion break plateaus in low collocation settings. Moreover, our methods are suitable for fractional derivatives. We establish that our methods improve spectral eigenvalue decay of the neural tangent kernel (NTK), and so our methods contribute towards the learning of high frequencies in early training, mitigating the effects of frequency bias up to the polynomial order and possibly greater with smooth activations. Our methods accommodate advanced techniques in PINNs, such as Fourier feature embeddings. A pitfall of discrete Fourier transforms via the Fast Fourier Transform (FFT) is mesh subjugation, and so we demonstrate compatibility of our methods for greater mesh flexibility and invariance on alternative Euclidean and non-Euclidean domains via Monte Carlo methods and otherwise.
