ART-VS: Adaptive Resolution Tiling for Vision Transformer Visual Servoing

2026-06-17

AI Summary

This work addresses the inherent trade-off in vision-based servoing between robustness and localization accuracy when using Vision Transformer (ViT) features: low-resolution features ensure stability but lack precision, while high-resolution features improve accuracy yet fail to enhance robustness. To resolve this, the authors propose a training-free, two-stage adaptive patch resolution strategy—first achieving stable coarse alignment at the original ViT resolution, followed by fine-grained localization via high-resolution patch matching within local neighborhoods. This approach introduces adaptive patching into ViT-based visual servoing for the first time and is compatible with various backbone networks. Experiments demonstrate a 95.4% convergence rate under perturbations, outperforming standard and full-resolution ViT by 18.8 and 14.4 percentage points, respectively; it also reduces localization error by 53%, accelerates inference by over 10×, cuts memory usage by 27%, and achieves success rates of 95/100 and 98/100 in transparent bottle and shoe grasping tasks.

Abstract

Visual servoing with self-supervised Vision Transformer (ViT) features enables training-free robotic positioning with strong generalization, but faces a fundamental trade-off between robustness and precision. Coarse patch-level descriptors provide stable correspondences yet limit positioning accuracy. Increasing image resolution improves precision but yields only marginal robustness gains - under perturbation, high-resolution processing improves convergence success rate from 76.6% to just 81.0% despite 12x more ViT patches. Therefore, we propose Adaptive Resolution Tiling Visual Servoing (ART-VS), a two-phase method that adapts feature granularity to servoing progress: a coarse phase at native ViT resolution for stable alignment, then a tiled high-resolution phase that restricts matching to local neighborhoods improving positioning accuracy. Without any task-specific training, ART-VS achieves 95.4% convergence under perturbation, outperforming standard and full-resolution ViT-based servoing by 18.8 and 14.4 percentage points. Over the former it reduces positioning error by 53%, while running at over 10x higher speed and 27% lower VRAM than the latter. We validate ART-VS across three ViT backbones and demonstrate real-world category-level grasping of unseen object instances, achieving 95/100 on transparent bottles and 98/100 on shoes. Code available under https://art-vs.github.io/.

Problem

• visual servoing
• Vision Transformer
• robustness-precision trade-off
• feature granularity
• self-supervised features

Innovation

• Adaptive Resolution Tiling
• Vision Transformer
• Visual Servoing
• Self-supervised Features
• Robustness-Precision Trade-off