PathFound: An Agentic Multimodal Model Activating Evidence-seeking Pathological Diagnosis

2025-12-29

AI Summary

Current pathology foundation models rely on static, single-pass inference, failing to emulate clinicians’ iterative diagnostic process involving repeated observation and progressive evidence refinement. To address this, we propose an agent-based multimodal model for pathology diagnosis, introducing a novel three-stage dynamic reasoning paradigm: “initial diagnosis → active evidence seeking → final decision.” Our framework integrates vision foundation models, vision-language models, and a reinforcement learning–driven reasoning module, enabling multi-scale localization, natural language–guided targeted retrieval, and policy optimization. It is the first pathology model to realize clinical reasoning–inspired adaptive multi-round visual focusing and cross-modal inference. This significantly enhances detection of subtle pathological features—e.g., nuclear atypia and microinvasion. Evaluated on multiple large-scale multimodal benchmarks, our method achieves state-of-the-art performance, delivering both high diagnostic accuracy and superior sensitivity for early-stage and occult lesions.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Computer Vision: Multi-modal Vision
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Recent pathological foundation models have substantially advanced visual representation learning and multimodal interaction. However, most models still rely on a static inference paradigm in which whole-slide images are processed once to produce predictions, without reassessment or targeted evidence acquisition under ambiguous diagnoses. This contrasts with clinical diagnostic workflows that refine hypotheses through repeated slide observations and further examination requests. We propose PathFound, an agentic multimodal model designed to support evidence-seeking inference in pathological diagnosis. PathFound integrates the power of pathological visual foundation models, vision-language models, and reasoning models trained with reinforcement learning to perform proactive information acquisition and diagnosis refinement by progressing through the initial diagnosis, evidence-seeking, and final decision stages. Across several large multimodal models, adopting this strategy consistently improves diagnostic accuracy, indicating the effectiveness of evidence-seeking workflows in computational pathology. Among these models, PathFound achieves state-of-the-art diagnostic performance across diverse clinical scenarios and demonstrates strong potential to discover subtle details, such as nuclear features and local invasions.

Problem

• Develops an agentic model for dynamic evidence-seeking in pathology
• Enhances diagnostic accuracy through proactive information acquisition
• Identifies subtle pathological details like nuclear features and invasions

Innovation

• Agentic multimodal model for evidence-seeking pathological diagnosis
• Integrates visual foundation, vision-language, and reinforcement learning models
• Proactive information acquisition through diagnosis refinement stages