Retrieval Augmented Decision-Making: A Requirements-Driven, Multi-Criteria Framework for Structured Decision Support

2025-05-24

AI Summary

Industrial documents—such as industrial development plans, technical guidelines, and regulatory texts—are structurally complex and semantically fragmented, posing significant challenges in information retrieval, comprehension, and transparent decision support for domain experts and policymakers. To address these issues, we propose RAD, a Retrieval-Augmented Decision-support framework that uniquely integrates Multi-Criteria Decision Making (MCDM) with large language model (LLM)-based semantic understanding. RAD’s core contributions are: (1) explicit criterion extraction and automated weight assignment; (2) traceable, hierarchical reasoning chains; and (3) structured, interpretable report generation. By synergizing information retrieval, MCDM modeling, prompt engineering, and structured output guidance, RAD significantly outperforms existing RAG approaches in granularity, logical coherence, and regulatory compliance. Empirical evaluation demonstrates its effectiveness in delivering transparent, reliable, and multi-level decision support within intricate industrial contexts.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Learning Preferences or Rankings

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Various industries have produced a large number of documents such as industrial plans, technical guidelines, and regulations that are structurally complex and content-wise fragmented. This poses significant challenges for experts and decision-makers in terms of retrieval and understanding. Although existing LLM-based Retrieval-Augmented Generation methods can provide context-related suggestions, they lack quantitative weighting and traceable reasoning paths, making it difficult to offer multi-level and transparent decision support. To address this issue, this paper proposes the RAD method, which integrates Multi-Criteria Decision Making with the semantic understanding capabilities of LLMs. The method automatically extracts key criteria from industry documents, builds a weighted hierarchical decision model, and generates structured reports under model guidance. The RAD framework introduces explicit weight assignment and reasoning chains in decision generation to ensure accuracy, completeness, and traceability. Experiments show that in various decision-making tasks, the decision reports generated by RAD significantly outperform existing methods in terms of detail, rationality, and structure, demonstrating its application value and potential in complex decision support scenarios.

Problem

• Addresses fragmented, complex industrial documents retrieval challenges
• Enhances decision support with quantitative weighting and traceable reasoning
• Integrates MCDM and LLMs for structured, transparent decision reports

Innovation

• Integrates Multi-Criteria Decision Making with LLMs
• Automatically extracts and weights key criteria
• Generates structured reports with traceable reasoning