Architecture Matters More Than Scale: A Comparative Study of Retrieval and Memory Augmentation for Financial QA Under SME Compute Constraints

2026-04-20

AI Summary

This study addresses the combined demands of cost-efficiency, accuracy, and regulatory compliance in financial question answering for small and medium-sized enterprises operating under constrained computational resources. It systematically evaluates four locally deployed 8B-parameter large language model inference architectures on the FinQA and ConvFinQA benchmarks. The findings indicate that structured memory mechanisms are better suited for deterministic numerical reasoning, whereas retrieval-augmented generation (RAG) excels in handling conversational implicit references. Building on these insights, the authors propose a hybrid inference framework that dynamically switches between memory-based and retrieval-based strategies. This approach significantly enhances numerical accuracy, auditability, and deployment efficiency without relying on cloud infrastructure, thereby offering a practical solution for deploying financial AI in resource-constrained environments.

Technology Category

• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The rapid adoption of artificial intelligence (AI) and large language models (LLMs) is transforming financial analytics by enabling natural language interfaces for reporting, decision support, and automated reasoning. However, limited empirical understanding exists regarding how different LLM-based reasoning architectures perform across realistic financial workflows, particularly under the cost, accuracy, and compliance constraints faced by small and medium-sized enterprises (SMEs). SMEs typically operate within severe infrastructure constraints, lacking cloud GPU budgets, dedicated AI teams, and API-scale inference capacity, making architectural efficiency a first-class concern. To ensure practical relevance, we introduce an explicit SME-constrained evaluation setting in which all experiments are conducted using a locally hosted 8B-parameter instruction-tuned model without cloud-scale infrastructure. This design isolates the impact of architectural choices within a realistic deployment environment. We systematically compare four reasoning architectures: baseline LLM, retrieval-augmented generation (RAG), structured long-term memory, and memory-augmented conversational reasoning across both FinQA and ConvFinQA benchmarks. Results reveal a consistent architectural inversion: structured memory improves precision in deterministic, operand-explicit tasks, while retrieval-based approaches outperform memory-centric methods in conversational, reference-implicit settings. Based on these findings, we propose a hybrid deployment framework that dynamically selects reasoning strategies to balance numerical accuracy, auditability, and infrastructure efficiency, providing a practical pathway for financial AI adoption in resource-constrained environments.

Problem

• financial QA
• SME compute constraints
• reasoning architectures
• retrieval-augmented generation
• memory augmentation

Innovation

• reasoning architecture
• retrieval-augmented generation
• structured memory
• SME-constrained AI
• hybrid deployment framework