From General Vision to Reliable Traversability Estimation: Adapting Vision Foundation Models for Unstructured Outdoor Environments

2026-05-28

AI Summary

Existing vision foundation model–based approaches to traversability estimation suffer from limited reliability due to task-agnostic designs, ambiguous annotations, and misalignment between semantic predictions and physical safety. To address these issues, this work proposes the ViTA framework, which extends SAM2 by introducing learnable traversability prompts to inject task-specific knowledge, employs multi-view training to mitigate boundary ambiguity, and leverages geometric knowledge distillation to infer slope and elevation from RGB images alone. By unifying task-aware prompting, semantic uncertainty modeling, and geometry-aware distillation, ViTA produces continuous traversability scores that effectively fuse semantic and geometric cues. The method achieves state-of-the-art performance on multiple real-world off-road datasets, demonstrating superior intersection-over-union and precision, lower false-positive rates, and strong cross-domain generalization in unstructured outdoor environments.

Abstract

Vision-based approaches have become the dominant paradigm for traversability estimation in unstructured outdoor environments, typically adapting vision foundation models (VFMs) via semantic segmentation supervision. However, this paradigm faces three fundamental challenges that undermine its reliability: the task-agnostic design of VFMs, the ambiguity of traversability annotations, and the discrepancy between semantic labels and physical safety. We propose Vision-to-Traversability Adaptation (ViTA), a framework that adapts VFMs for reliable traversability estimation, instantiated on SAM2. ViTA injects task-specific knowledge through learnable traversability prompts while preserving the VFM's cross-domain generalization. To handle annotation ambiguity, we introduce Perspective-Diversified Training, which estimates semantic uncertainty to suppress confident predictions at ambiguous boundaries. To bridge the semantic-traversability discrepancy, we distill geometric knowledge during training, enabling slope and elevation reasoning from RGB images alone at inference. The semantic and geometric outputs are fused into a continuous traversability score that reflects both semantic uncertainty and geometric risk. Evaluations across diverse domains, including challenging real-world off-road datasets, demonstrate that ViTA achieves state-of-the-art IoU and Precision with substantial false-positive reduction and strong cross-domain generalization.

Problem

• traversability estimation
• vision foundation models
• unstructured outdoor environments
• semantic ambiguity
• semantic-physical discrepancy

Innovation

• Vision Foundation Models
• Traversability Estimation
• Geometric Knowledge Distillation
• Semantic Uncertainty
• Cross-domain Generalization