Interactive Symbolic Regression through Offline Reinforcement Learning: A Co-Design Framework

2024-02-07

AI Summary

Symbolic regression faces challenges including an exponentially large expression search space, insufficient integration of domain expertise, and lack of real-time human-in-the-loop interaction. Method: This paper proposes Sym-Q, a symbolic regression framework based on offline reinforcement learning. It introduces a novel non-Transformer tree-structured encoder that models symbolic expressions as an editable action space, and incorporates a human–machine co-design mechanism enabling domain experts to dynamically edit nodes and embed physical constraints at any stage. Contribution/Results: Unlike purely data-driven or black-box interactive approaches, Sym-Q achieves significant performance gains over state-of-the-art symbolic regression models on the SSDNC benchmark. In real-world physics modeling tasks, iterative expert interaction improves equation discovery accuracy by up to 12.7%, outperforming all existing SOTA methods.

Technology Category

• Cognitive Modeling & Cognitive Systems: Symbolic Representations
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Neuro-Symbolic Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Symbolic Regression (SR) holds great potential for uncovering underlying mathematical and physical relationships from observed data. However, the vast combinatorial space of possible expressions poses significant challenges for both online search methods and pre-trained transformer models. Additionally, current state-of-the-art approaches typically do not consider the integration of domain experts' prior knowledge and do not support iterative interactions with the model during the equation discovery process. To address these challenges, we propose the Symbolic Q-network (Sym-Q), an advanced interactive framework for large-scale symbolic regression. Unlike previous large-scale transformer-based SR approaches, Sym-Q leverages reinforcement learning without relying on a transformer-based decoder. This formulation allows the agent to learn through offline reinforcement learning using any type of tree encoder, enabling more efficient training and inference. Furthermore, we propose a co-design mechanism, where the reinforcement learning-based Sym-Q facilitates effective interaction with domain experts at any stage of the equation discovery process. Users can dynamically modify generated nodes of the expression, collaborating with the agent to tailor the mathematical expression to best fit the problem and align with the assumed physical laws, particularly when there is prior partial knowledge of the expected behavior. Our experiments demonstrate that the pre-trained Sym-Q surpasses existing SR algorithms on the challenging SSDNC benchmark. Moreover, we experimentally show on real-world cases that its performance can be further enhanced by the interactive co-design mechanism, with Sym-Q achieving greater performance gains than other state-of-the-art models. Our reproducible code is available at https://github.com/EPFL-IMOS/Sym-Q.

Problem

• Enhance symbolic regression efficiency
• Integrate expert knowledge interactively
• Optimize equation discovery process

Innovation

• Offline reinforcement learning
• Interactive co-design mechanism
• Tree encoder flexibility