InsightX Agent: An LMM-based Agentic Framework with Integrated Tools for Reliable X-ray NDT Analysis

2025-07-20

AI Summary

Current deep learning methods for X-ray non-destructive testing suffer from weak interactivity, poor interpretability, and lack of self-assessment capability, limiting their industrial trustworthiness. To address these issues, we propose an intelligent agent framework centered on a large vision model (LVM), introducing the novel Evidence-Guided Reflection (EGR) mechanism to enable closed-loop reasoning encompassing defect detection, logical verification, and confidence calibration. Methodologically, we design a Sparse Deformable Multi-Scale Detector (SDMSD) to enhance small-object detection sensitivity and integrate an optimized non-maximum suppression strategy for high-quality candidate proposal generation. Evaluated on the GDXray+ dataset, our approach achieves an F1-score of 96.35%, substantially outperforming existing baselines. Moreover, it provides traceable decision rationales and an interactive analytical interface, jointly ensuring high accuracy and strong interpretability—thereby significantly improving human-AI collaboration reliability and operator trust.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Nonmonotonic Reasoning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Non-destructive testing (NDT), particularly X-ray inspection, is vital for industrial quality assurance, yet existing deep-learning-based approaches often lack interactivity, interpretability, and the capacity for critical self-assessment, limiting their reliability and operator trust. To address these shortcomings, this paper proposes InsightX Agent, a novel LMM-based agentic framework designed to deliver reliable, interpretable, and interactive X-ray NDT analysis. Unlike typical sequential pipelines, InsightX Agent positions a Large Multimodal Model (LMM) as a central orchestrator, coordinating between the Sparse Deformable Multi-Scale Detector (SDMSD) and the Evidence-Grounded Reflection (EGR) tool. The SDMSD generates dense defect region proposals for multi-scale feature maps and sparsifies them through Non-Maximum Suppression (NMS), optimizing detection of small, dense targets in X-ray images while maintaining computational efficiency. The EGR tool guides the LMM agent through a chain-of-thought-inspired review process, incorporating context assessment, individual defect analysis, false positive elimination, confidence recalibration and quality assurance to validate and refine the SDMSD's initial proposals. By strategically employing and intelligently using tools, InsightX Agent moves beyond passive data processing to active reasoning, enhancing diagnostic reliability and providing interpretations that integrate diverse information sources. Experimental evaluations on the GDXray+ dataset demonstrate that InsightX Agent not only achieves a high object detection F1-score of 96.35% but also offers significantly improved interpretability and trustworthiness in its analyses, highlighting the transformative potential of agentic LLM frameworks for industrial inspection tasks.

Problem

• Enhances X-ray NDT analysis reliability and interpretability
• Integrates interactive tools for defect detection and validation
• Improves operator trust through active reasoning and self-assessment

Innovation

• LMM orchestrates SDMSD and EGR tools
• SDMSD optimizes small defect detection
• EGR tool refines proposals via multi-step review