GramLoop: Training-Free Gram-Gated Replay for Robust Dense Prediction

2026-08-29

AI Summary

为提高DINOv3模型在分布偏移下的预测鲁棒性，提出GramLoop方法，在不改变模型权重的情况下，通过重放变换器窗口并利用余弦-Gram一致性控制来优化特征。

Abstract

We aim to improve frozen DINOv3 dense-prediction models under distribution shift by adding inference computation inside the visual backbone, without changing model weights, task adapters, or prediction heads. The challenge is that repeated transformer-block computation must refine dense features without disrupting the pairwise patch relations that DINOv3 uses to preserve spatial structure. We introduce GramLoop, a training-free framework that replays a short transformer window and controls each replay through final-layer cosine-Gram consistency. Each proposal is propagated through the frozen suffix, measured against the standard DINOv3 trajectory, and accepted through a patchwise gate at the replay-window endpoint. Across object detection and semantic segmentation under corruptions, perturbations, and natural shifts, GramLoop improves all five shifted benchmarks over the paired DINOv3 baseline. On COCO-O, it improves mAP by +0.252 and Effective Robustness by +0.250, while preserving clean ADE20K performance. Code will be released.

Problem

• distribution shift
• frozen DINOv3
• dense prediction
• visual backbone
• transformer-block computation

Innovation

• Training-Free
• Gram-Gated Replay
• Dense Prediction
• Transformer-Block Computation
• Cosine-Gram Consistency