Language-Guided Representation Learning for Robust Cross-Sensor Material Recognition

2026-09-13

AI Summary

为解决跨传感器材料识别的鲁棒性问题，提出了一种语言引导的表示学习方法，通过触觉-语言数据集训练模型以提高准确性和泛化能力。

Abstract

Robots need touch to manipulate objects safely and reliably, as many properties, such as softness, texture, and contact stability, are hard to infer from vision alone. However, vision-based tactile sensors yield different observations of the same material due to variations in optics, elastomer properties, and illumination, leading to poor generalization when trained on a single or multiple sensors. We propose a language-guided distillation framework for learning sensor-robust tactile representations. Language encodes high-level semantic properties of touch (e.g., rough, soft, slippery) that remain invariant across sensing hardware, providing a natural sensor-agnostic supervisory signal. We construct a 39K-sample touch-language dataset with human-annotated material labels and train a tactile encoder to align sensor-specific tactile images with language embeddings in a shared semantic space. We evaluate our approach for few-shot learning and cross-sensor transfer and benchmark it on six existing tactile datasets. Our method achieves 95% accuracy in the 100-shot setting, improves cross-sensor transfer by an average of 13.3% accuracy, and yields up to 19% accuracy gains across six existing tactile datasets. These results demonstrate that language-guided distillation enables scalable and hardware-agnostic tactile representation learning. Code and dataset are available at https://mashood3624.github.io/Language_Tactile/

Problem

• tactile sensors
• material recognition
• cross-sensor

Innovation

• language-guided distillation
• sensor-robust tactile representations
• touch-language dataset
• cross-sensor transfer
• few-shot learning