VBT-MPC: Vision-Based Tactile MPC for Contour Following

2026-05-19

AI Summary

This work addresses the challenge of simultaneously maintaining contact and accurately tracking object contours in robotic contour-following tasks by proposing a vision-based tactile model predictive control framework (VBT-MPC). For the first time, model predictive control is directly applied in the contour feature space extracted from an eye-in-hand visuotactile sensor, eliminating the need for separate pose estimation or complex force-control modules. By integrating visuotactile perception with feature-based visual servoing, VBT-MPC achieves high-precision and stable contour tracking across objects with diverse geometries and material properties in both simulation and real-world experiments. This approach substantially simplifies the system architecture while significantly enhancing tracking performance.

Abstract

Tactile sensing plays a key role in robotic manipulation, particularly in tasks like surface inspection. Successful execution requires maintaining contact while accurately tracking object contours. In this work, we propose a Vision-Based Tactile Model Predictive Control (VBT-MPC) framework for robotic contour following using a Vision-Based Tactile Sensor (VBTS) mounted in an eye-in-hand configuration. The proposed controller operates directly in contour features space, thereby avoiding the need for separate pose-estimation modules or complex force-control architectures. We further compare our VBT-MPC with visual-servoing strategies adapted to tactile features, and evaluate contour tracking on objects with diverse geometries and materials in both simulation and real-world experiments.

Problem

• tactile sensing
• contour following
• robotic manipulation
• surface inspection
• contact maintenance

Innovation

• Vision-Based Tactile Sensing
• Model Predictive Control
• Contour Following
• Tactile Servoing
• Eye-in-Hand Configuration