OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systems

2026-06-17

AI Summary

This work addresses the redundancy and reduced interpretability commonly observed in hybrid neuro-symbolic dynamical systems, where neural and symbolic components often exhibit functional overlap. To mitigate this issue, the authors propose Orthogonal Regularization (OrthoReg), a method that enforces orthogonality between neural residuals and symbolic components, thereby achieving a complementary decomposition of their respective contributions. OrthoReg remains effective even when the underlying symbolic structure must be discovered from data and the available symbolic library is incomplete, successfully preventing the neural network from subsuming symbolic mechanisms. Experimental results demonstrate that OrthoReg significantly improves the accuracy of symbolic recovery and enhances out-of-distribution predictive performance, leading to models with greater interpretability and generalization capability.

Abstract

Dynamical systems are fundamental to modeling the natural world, yet modeling them involves a persistent trade-off: manually prescribed mechanistic models are interpretable by design but often overly simplistic and misspecified; in contrast, flexible data-driven neural methods lack physical insight. Hybrid modeling aims for the best of both worlds by combining a prescribed or symbolic, physics-based component with a flexible neural network. A critical challenge, however, is that the neural component may relearn mechanistic parts, yielding redundant and uninterpretable models, especially when the symbolic structure itself is discovered from data. Existing methods based on standard $L^2$ regularization rely on a projection argument that breaks when the symbolic component is learned through sparse discovery, allowing the neural augmentation to overlap with symbolic structure. We introduce \textbf{OrthoReg} (Orthogonal Regularization), which directly penalizes overlap between the symbolic and neural components, preventing symbolic structure from being absorbed by the neural residual. This yields a complementary decomposition: the symbolic part captures what the library can express, and the neural part captures what remains. On benchmark dynamical systems with partial library mismatch, OrthoReg improves symbolic recovery and out-of-distribution behavior.

Problem

• hybrid modeling
• symbolic-neural systems
• orthogonal regularization
• dynamical systems
• model interpretability

Innovation

• Orthogonal Regularization
• Hybrid Modeling
• Symbolic-Neural Systems
• Dynamical Systems
• Sparse Discovery