Problem
Research questions and friction points this paper is trying to address.
Wild Test-Time Adaptation
Self-Referential Loop
Distribution Shifts
Sparse MLLM Anchors, Dense Adaptation: Breaking the Self-Referential Loop in Wild Test-Time Adaptation
🤖 AI Summary
为解决野生测试时适应中模型自参照循环问题，提出MASA方法，利用多模态大语言模型提供结构化语义描述辅助适应。
📝 Abstract
Wild test-time adaptation (WTTA) updates a source model online under small test batches, concurrent distribution shifts, and time-varying class imbalance. Most WTTA methods derive their adaptation signals, including predictive uncertainty, sample reliability, and local feature geometry, from the model being adapted. When the source model is unreliable under shift, these signals can reinforce its own errors, forming a self-referential loop. We introduce MASA (Multimodal-LLM-Anchored Semantic Adaptation), which complements model-internal evidence with structured semantic descriptions from a frozen multimodal large language model (MLLM). To limit inference cost, MASA queries the MLLM only for a small set of diverse, reliability-ranked anchors. The resulting descriptions capture the object family and nuisance factors such as style, viewpoint, and occlusion. MASA encodes these descriptions, propagates them to neighboring test samples, and stores the resulting visual-semantic information in an online prototype memory. Descriptor-aware retrieval from this memory provides an auxiliary target for lightweight adaptation of normalization-affine parameters. We evaluate MASA on the WTTA ImageNet-C benchmark under limited-batch, mixed-domain, and imbalanced-label-shift settings with ResNet and ViT backbones.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Multimodal-LLM-Anchored Semantic Adaptation
Wild Test-Time Adaptation
Structured Semantic Descriptions
Online Prototype Memory
Normalization-Affine Parameters
🔎 Similar Papers
No similar papers found.
