Triple Expert Learning from Noisy Labels for Semi-Supervised Vision Foundation Model Adaptation

2026-08-09

AI Summary

This work addresses the challenge of performance degradation in semi-supervised adaptation of vision foundation models caused by noise in pseudo-labels. To mitigate this issue, the authors propose TriNoL, a novel framework that introduces, for the first time, a three-expert LoRA mechanism. Unlabeled samples are partitioned into high-, medium-, and low-confidence groups based on pseudo-label confidence scores, and are respectively processed by positive, aligned, and negative LoRA experts. This differentiated treatment enables robust adaptation while keeping the backbone network frozen. The proposed method significantly enhances semi-supervised adaptation performance and effectively improves robustness against pseudo-label noise, all while maintaining low training overhead.

Abstract

Semi-supervised adaptation of vision foundation models (VFMs) commonly freezes the pretrained backbone and updates lightweight modules such as LoRA. However, pseudo-labels have mixed reliability, and a single LoRA adapter must absorb reliable, ambiguous, and noisy gradients in the same low-rank space. This can make VFM adaptation sensitive to pseudo-label noise. We propose \textbf{TriNoL}, a \textbf{Tri}ple-expert learning framework from \textbf{No}isy \textbf{L}abels for semi-supervised VFM adaptation. TriNoL routes unlabeled samples into three confidence regions and assigns them to three LoRA experts: a Positive Expert for high-confidence pseudo-labels, an Alignment Expert for medium-confidence ambiguous samples, and a Negative Expert for low-confidence noisy samples. The VFM backbone remains frozen, and only the LoRA experts and classifier head are updated. By separating different pseudo-label reliability regions into specialized adaptation paths, TriNoL improves robustness to noisy supervision while keeping the training cost low.

Problem

• noisy labels
• semi-supervised learning
• vision foundation models
• pseudo-label reliability
• model adaptation

Innovation

• Triple-expert learning
• Noisy pseudo-labels
• Semi-supervised adaptation
• LoRA
• Vision foundation models