Prompting Neural-Guided Equation Discovery Based on Residuals

2025-11-05

IFIP Working Conference on Database Semantics

AI Summary

Current neural-guided equation discovery systems lack efficient iterative optimization mechanisms when initial predictions fail to meet user requirements. To address this, we propose RED, a residual-based post-processing method: it parses the predicted expression into a syntax tree to localize subexpressions, computes node-level residuals to identify erroneous components, and generates targeted prompts to guide iterative refinement of critical subexpressions. RED is the first approach to incorporate residual analysis into prompt engineering for equation discovery; it requires no model retraining and is plug-and-play compatible with arbitrary neural-guided or classical genetic programming systems. Experiments on the Feynman benchmark—comprising 53 physical equations—demonstrate that RED significantly improves both accuracy and convergence speed across multiple baseline systems. The method exhibits strong generality, flexibility, and practical utility.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Feature Construction/Reformulation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Neural-guided equation discovery systems use a data set as prompt and predict an equation that describes the data set without extensive search. However, if the equation does not meet the user's expectations, there are few options for getting other equation suggestions without intensive work with the system. To fill this gap, we propose Residuals for Equation Discovery (RED), a post-processing method that improves a given equation in a targeted manner, based on its residuals. By parsing the initial equation to a syntax tree, we can use node-based calculation rules to compute the residual for each subequation of the initial equation. It is then possible to use this residual as new target variable in the original data set and generate a new prompt. If, with the new prompt, the equation discovery system suggests a subequation better than the old subequation on a validation set, we replace the latter by the former. RED is usable with any equation discovery system, is fast to calculate, and is easy to extend for new mathematical operations. In experiments on 53 equations from the Feynman benchmark, we show that it not only helps to improve all tested neural-guided systems, but also all tested classical genetic programming systems.

Problem

• Improving equation discovery systems through targeted residual-based post-processing
• Enhancing user control by generating better equation suggestions without intensive search
• Extending equation discovery capabilities for both neural and classical systems

Innovation

• Uses residuals to improve initial equations
• Applies node-based calculation on syntax trees
• Generates new prompts for equation refinement