Evaluating Few-Shot Pill Recognition Under Visual Domain Shift

2026-03-11

AI Summary

This study addresses the performance degradation in few-shot pill recognition under real-world conditions caused by visual domain shifts such as cluttered backgrounds, pill overlap, and specular reflections. The authors propose a two-stage fine-tuning approach based on an object detection framework: models are first pretrained on a base dataset and then fine-tuned with only 1, 5, or 10 samples per class. Emphasizing the critical role of visual realism in deployment robustness, the work treats few-shot fine-tuning as a diagnostic tool for assessing model readiness. Experiments reveal a decoupling between classification and localization performance under challenging conditions: semantic classification saturates even with a single example, whereas localization accuracy and recall drop significantly in occluded scenarios. Notably, models trained on realistic multi-pill scenes demonstrate superior robustness in low-data regimes.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Adverse drug events are a significant source of preventable harm, which has led to the development of automated pill recognition systems to enhance medication safety. Real-world deployment of these systems is hindered by visually complex conditions, including cluttered scenes, overlapping pills, reflections, and diverse acquisition environments. This study investigates few-shot pill recognition from a deployment-oriented perspective, prioritizing generalization under realistic cross-dataset domain shifts over architectural innovation. A two-stage object detection framework is employed, involving base training followed by few-shot fine-tuning. Models are adapted to novel pill classes using one, five, or ten labeled examples per class and are evaluated on a separate deployment dataset featuring multi-object, cluttered scenes. The evaluation focuses on classification-centric and error-based metrics to address heterogeneous annotation strategies. Findings indicate that semantic pill recognition adapts rapidly with few-shot supervision, with classification performance reaching saturation even with a single labeled example. However, stress testing under overlapping and occluded conditions demonstrates a marked decline in localization and recall, despite robust semantic classification. Models trained on visually realistic, multi-pill data consistently exhibit greater robustness in low-shot scenarios, underscoring the importance of training data realism and the diagnostic utility of few-shot fine-tuning for deployment readiness.

Problem

• few-shot learning
• pill recognition
• domain shift
• object detection
• medication safety

Innovation

• few-shot learning
• pill recognition
• domain shift
• object detection
• deployment robustness