Towards Expert Financial QA via Self-Improving RAG

2026-08-27

AI Summary

研究通过自改进RAG框架，利用三个专门代理和反馈驱动的自我纠正机制解决金融问答中的数据验证和合规问题，提高了答案准确性和可解释性。

Abstract

Expert-level financial question answering requires both grounded verification to catch numeric hallucinations and audit trails for regulatory compliance, attributes that standard single-pass RAG systems lack. We take a step toward this goal with Self-Improving RAG, a framework that decomposes document QA into three specialized agents (Retrieval, Reasoning, and Judge) coordinated by an orchestrator with feedback-driven self-correction. When the Judge Agent scores an answer below a dynamic threshold, the system triggers retry with escalated strategies: broader retrieval, more careful prompting, and relaxed acceptance criteria. We evaluate on FinanceBench (SEC filing QA), where Self-Improving RAG achieves 86% oracle-guided accuracy (measuring agreement with gold answers) with a 36.4% Lazarus Rate, recovering nearly 4 in 10 initially incorrect answers through targeted retry. A key finding is that a fixed retrieval pipeline with judge-driven retry achieves strong results without dynamic routing, providing full interpretability. Every decision is logged with confidence scores, enabling the audit trails required for regulated financial applications.

Problem

• Financial QA
• Numeric Hallucinations
• Regulatory Compliance

Innovation

• Self-Improving RAG
• feedback-driven self-correction
• dynamic threshold
• Lazarus Rate
• audit trails