Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to quantify the effective degrees of freedom in Physics-Informed Neural Networks (PINNs) using the Fisher Information Matrix, addressing the issue of task interference caused by shared parameter spaces. The proposed effective dimensionality ($d_{eff}$) serves as a structural invariant that accurately reflects the kernel dimension of differential operators, independent of network architecture, and provides a diagnostic tool for assessing the absorption of free directions by physics constraints. Additionally, the authors present subspace projection strategies for boundary adaptation that allow for efficient updates to boundary conditions without retraining, achieving comparable performance to gradient-based methods in significantly less time.
The effective dimensionality $d_{eff}$ reveals how well physics constraints absorb network degrees of freedom, transforming our understanding of PINN performance and adaptation.
Physics-Informed Neural Networks inherently suffer from task interference because they rely on a shared parameter space to satisfy both governing differential equations and boundary conditions. We analyze this structural conflict using the Fisher Information Matrix to quantify the effective degrees of freedom ($d_{eff}$) in a physics-constrained model. Unlike the classical $d_{eff}$ which measures how many parameter directions are informed by data against a statistical prior, our $d_{eff}$ measures the dimension of the parameter directions unconstrained by the differential operator. For operators with finite-dimensional kernel, we show that $d_{eff}$ converges to the kernel dimension exactly, independent of network width, depth, or activation function, recasting it from a fit diagnostic into a structural invariant of the underlying continuous operator. For operators with infinite-dimensional kernel, $d_{eff}$ instead measures the network's finite-dimensional representational bandwidth for that kernel rather than recovering an integer invariant. Importantly, $d_{eff}$ also serves as an a priori structural diagnostic. Driving $d_{eff}$ of a well-posed problem to zero certifies that the physics and boundary constraints have absorbed the network's free directions. Building on this characterization, we introduce subspace projection strategies for boundary adaptation. Rather than retraining from scratch, we project parameter updates into the null space of the pre-trained physics operator so that new boundary conditions are satisfied without disturbing the learned physics. Gradient-based fine-tuning can match or exceed this but needs more wall-clock time and tuning, whereas subspace projection delivers near-equivalent quality in seconds to minutes. We validate on linear and nonlinear operators, demonstrating accurate adaptation to initial and boundary shifts and unencountered constraint types.
Johns Hopkins