Search papers, labs, and topics across Lattice.
This paper introduces OrthoReg, an orthogonal regularization technique designed to enhance hybrid symbolic-neural dynamical systems by preventing the neural component from redundantly learning mechanistic aspects of the symbolic model. The method addresses the limitations of existing regularization approaches that fail when symbolic structures are learned from data, ensuring that the symbolic and neural components remain distinct and interpretable. Experimental results demonstrate that OrthoReg significantly improves symbolic recovery and out-of-distribution performance on benchmark dynamical systems with partial library mismatches.
OrthoReg prevents neural networks from absorbing symbolic structures, leading to clearer and more interpretable hybrid models in dynamical systems.
Dynamical systems are fundamental to modeling the natural world, yet modeling them involves a persistent trade-off: manually prescribed mechanistic models are interpretable by design but often overly simplistic and misspecified; in contrast, flexible data-driven neural methods lack physical insight. Hybrid modeling aims for the best of both worlds by combining a prescribed or symbolic, physics-based component with a flexible neural network. A critical challenge, however, is that the neural component may relearn mechanistic parts, yielding redundant and uninterpretable models, especially when the symbolic structure itself is discovered from data. Existing methods based on standard $L^2$ regularization rely on a projection argument that breaks when the symbolic component is learned through sparse discovery, allowing the neural augmentation to overlap with symbolic structure. We introduce \textbf{OrthoReg} (Orthogonal Regularization), which directly penalizes overlap between the symbolic and neural components, preventing symbolic structure from being absorbed by the neural residual. This yields a complementary decomposition: the symbolic part captures what the library can express, and the neural part captures what remains. On benchmark dynamical systems with partial library mismatch, OrthoReg improves symbolic recovery and out-of-distribution behavior.
