SAM3-Adapter: Efficient Adaptation of Segment Anything 3 for Camouflage Object Segmentation, Shadow Detection, and Medical Image Segmentation

2025-11-24

AI Summary

Existing foundation models (e.g., SAM/SAM2) exhibit limited generalization and insufficient accuracy on fine-grained low-level vision tasks such as camouflaged object detection, shadow detection, and medical image segmentation. To address this, we propose SAM3-Adapter—the first lightweight adapter framework tailored for Segment Anything 3 (SAM3)—which enhances fine structural modeling while preserving SAM3’s strong generalization capability. Our approach introduces modular, parameter-efficient adapters coupled with task-aware training strategies, enabling unified multi-task segmentation with minimal computational overhead and flexible deployment. Evaluated on four challenging fine-grained segmentation benchmarks, SAM3-Adapter consistently outperforms SAM/SAM2 and their existing adapter variants, achieving state-of-the-art performance. Comprehensive experiments demonstrate its superior accuracy, robustness, and efficiency, validating its effectiveness for demanding low-level vision applications.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The rapid rise of large-scale foundation models has reshaped the landscape of image segmentation, with models such as Segment Anything achieving unprecedented versatility across diverse vision tasks. However, previous generations-including SAM and its successor-still struggle with fine-grained, low-level segmentation challenges such as camouflaged object detection, medical image segmentation, cell image segmentation, and shadow detection. To address these limitations, we originally proposed SAM-Adapter in 2023, demonstrating substantial gains on these difficult scenarios. With the emergence of Segment Anything 3 (SAM3)-a more efficient and higher-performing evolution with a redesigned architecture and improved training pipeline-we revisit these long-standing challenges. In this work, we present SAM3-Adapter, the first adapter framework tailored for SAM3 that unlocks its full segmentation capability. SAM3-Adapter not only reduces computational overhead but also consistently surpasses both SAM and SAM2-based solutions, establishing new state-of-the-art results across multiple downstream tasks, including medical imaging, camouflaged (concealed) object segmentation, and shadow detection. Built upon the modular and composable design philosophy of the original SAM-Adapter, SAM3-Adapter provides stronger generalizability, richer task adaptability, and significantly improved segmentation precision. Extensive experiments confirm that integrating SAM3 with our adapter yields superior accuracy, robustness, and efficiency compared to all prior SAM-based adaptations. We hope SAM3-Adapter can serve as a foundation for future research and practical segmentation applications. Code, pre-trained models, and data processing pipelines are available.

Problem

• Enhancing SAM3 for fine-grained segmentation tasks like camouflaged objects
• Improving segmentation accuracy in medical imaging and shadow detection
• Reducing computational overhead while boosting model adaptability and precision

Innovation

• Adapter framework tailored for Segment Anything 3
• Reduces computational overhead while improving performance
• Enhances segmentation precision across multiple downstream tasks