DentalGPT: Incentivizing Multimodal Complex Reasoning in Dentistry

2025-12-12

AI Summary

To address two key bottlenecks of existing multimodal large language models (MLLMs) in dentistry—limited fine-grained visual understanding and insufficient clinical reasoning—this work introduces the first high-quality dental multimodal dataset comprising over 120,000 image-text pairs. We propose a domain-specific two-stage training paradigm: (1) knowledge-informed supervised fine-tuning for cross-modal alignment, followed by (2) clinical-feedback-driven reinforcement learning from human feedback (RLHF) to enhance diagnostic reasoning. The resulting lightweight (7B-parameter) dental MLLM achieves substantial improvements in fine-grained interpretation of panoramic radiographs and intraoral images, as well as in complex disease discrimination. It consistently outperforms general-purpose MLLMs—including LLaVA and Qwen-VL—on dental visual question answering (VQA) and disease classification benchmarks, setting new state-of-the-art performance across multiple metrics. This work establishes both a reusable methodology and a foundational benchmark resource for vertical-domain MLLM development.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Reliable interpretation of multimodal data in dentistry is essential for automated oral healthcare, yet current multimodal large language models (MLLMs) struggle to capture fine-grained dental visual details and lack sufficient reasoning ability for precise diagnosis. To address these limitations, we present DentalGPT, a specialized dental MLLM developed through high-quality domain knowledge injection and reinforcement learning. Specifically, the largest annotated multimodal dataset for dentistry to date was constructed by aggregating over 120k dental images paired with detailed descriptions that highlight diagnostically relevant visual features, making it the multimodal dataset with the most extensive collection of dental images to date. Training on this dataset significantly enhances the MLLM's visual understanding of dental conditions, while the subsequent reinforcement learning stage further strengthens its capability for multimodal complex reasoning. Comprehensive evaluations on intraoral and panoramic benchmarks, along with dental subsets of medical VQA benchmarks, show that DentalGPT achieves superior performance in disease classification and dental VQA tasks, outperforming many state-of-the-art MLLMs despite having only 7B parameters. These results demonstrate that high-quality dental data combined with staged adaptation provides an effective pathway for building capable and domain-specialized dental MLLMs.

Problem

• Develops a specialized dental MLLM for precise diagnosis
• Enhances visual understanding and reasoning with dental data
• Improves disease classification and VQA in dentistry benchmarks

Innovation

• High-quality domain knowledge injection enhances visual understanding
• Reinforcement learning strengthens multimodal complex reasoning capabilities
• Staged adaptation with specialized data builds domain-specific MLLMs effectively