One More Question is Enough, Expert Question Decomposition (EQD) Model for Domain Quantitative Reasoning

2025-10-01

AI Summary

Domain-specific quantitative reasoning remains a critical bottleneck for large language models (LLMs), particularly in complex question-answering tasks requiring deep expertise—such as those in finance. To address this, we propose the Expert Question Decomposition (EQD) model. Methodologically, EQD leverages two key insights: (1) generating a single high-quality subquestion is more effective than multi-step guided decomposition; and (2) a two-stage fine-tuning framework with an efficacy-driven reward function enables efficient, low-resource decomposition. Training requires only minimal annotated data and a single A100 GPU, while inference latency is comparable to zero-shot prompting. Evaluated on four financial benchmark datasets, EQD consistently improves QA accuracy by 0.6–10.5 percentage points across multiple mainstream LLMs, significantly outperforming both domain-specific models and state-of-the-art prompting techniques.

Technology Category

• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web question answering
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Domain-specific quantitative reasoning remains a major challenge for large language models (LLMs), especially in fields requiring expert knowledge and complex question answering (QA). In this work, we propose Expert Question Decomposition (EQD), an approach designed to balance the use of domain knowledge with computational efficiency. EQD is built on a two-step fine-tuning framework and guided by a reward function that measures the effectiveness of generated sub-questions in improving QA outcomes. It requires only a few thousand training examples and a single A100 GPU for fine-tuning, with inference time comparable to zero-shot prompting. Beyond its efficiency, EQD outperforms state-of-the-art domain-tuned models and advanced prompting strategies. We evaluate EQD in the financial domain, characterized by specialized knowledge and complex quantitative reasoning, across four benchmark datasets. Our method consistently improves QA performance by 0.6% to 10.5% across different LLMs. Our analysis reveals an important insight: in domain-specific QA, a single supporting question often provides greater benefit than detailed guidance steps.

Problem

• Addressing domain-specific quantitative reasoning challenges in large language models
• Balancing domain knowledge usage with computational efficiency requirements
• Improving complex question answering through expert question decomposition

Innovation

• Two-step fine-tuning framework for question decomposition
• Reward function optimizing sub-question effectiveness
• Single GPU training with minimal data requirements