🤖 AI Summary
Traditional RAG systems struggle to extract reliable, verifiable insights from financial documents containing mixed formats and integrate them into analysts’ workflows. This work proposes a multi-agent RAG framework that combines structure-preserving PDF parsing, table-aware chunking, metadata enrichment, agent-driven query planning, and hybrid retrieval, augmented with a context generation mechanism capable of numerical reasoning. Evaluated on FinanceBench, the approach achieves an accuracy of 89.3%, substantially outperforming baseline methods. Furthermore, assessments by four professional financial analysts confirm its high accuracy, usability, and practical deployment value in real-world analytical tasks.
📝 Abstract
Retrieval-augmented generation (RAG) systems offer a promising approach to reduce hallucinations and improve answer accuracy in large language models (LLMs), a requirement for reliable, financial analysis where answers must be grounded in verifiable evidence from filings rather than generated from model priors. However, designing RAG systems that extract meaningful insights from mixed financial documents and integrate into analyst workflows remains challenging. This paper introduces MimirRAG (Metadata-Integrated Multi-Agent Information Retrieval), a multi-agent RAG system developed iteratively to address these challenges. MimirRAG features a modular pipeline encompassing structure-preserving parsing of PDF filings, table-aware chunking, metadata extraction, agent-based retrieval with query planning and hybrid search, validation, and context-aware generation with numerical reasoning support. Our ablation study identifies three key technical enablers for effective financial RAG: metadata integration, table-aware chunking, and an agentic workflow. MimirRAG was evaluated quantitatively using FinanceBench and qualitatively through expert validation with four financial analysts. The system achieved 89.3% accuracy on FinanceBench, outperforming the original benchmark baselines. Expert feedback highlighted that successful deployment also requires calibrated trust, comprehensive data integration, and user personalization. We conclude that combining multi-agent RAG architecture with human-centric design principles can improve the extraction of meaningful insights in financial analysis.