MiniTac: An Ultra-Compact $ ext{8 mm}$ Vision-Based Tactile Sensor for Enhanced Palpation in Robot-Assisted Minimally Invasive Surgery

2024-10-30

IEEE Robotics and Automation Letters

AI Summary

In robot-assisted minimally invasive surgery (RAMIS), the absence of haptic feedback severely impairs intraoperative tissue discrimination and detection of occult lesions. To address this, we present MiniTac—a miniature vision-based tactile sensor (8 mm diameter)—that integrates a mechanically responsive photonic elastomer film into a compact embedded imaging system. Pressure-induced structural color shifts enable high-sensitivity, real-time, and visually interpretable tactile sensing. MiniTac is fully compatible with the da Vinci surgical system without requiring additional trocars. In phantom and ex vivo tissue experiments, it successfully identified both superficial and deep-seated (>5 mm) tumor regions with a pressure resolution of 0.1 kPa and a color-pressure mapping model error <3%. This work overcomes the critical haptic deficit in RAMIS by introducing the first clinically integrable, miniaturized, and visualization-enabled vision-based tactile sensing solution for minimally invasive surgery.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Motion & Tracking
• Humans and AI: Brain-Sensing and Analysis

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Robot-assisted minimally invasive surgery (RAMIS) provides substantial benefits over traditional open and laparoscopic methods. However, a significant limitation of robot-assisted minimally invasive surgery (RAMIS) is the surgeon's inability to palpate tissues, a crucial technique for examining tissue properties and detecting abnormalities, restricting the widespread adoption of RAMIS. To overcome this obstacle, we introduce MiniTac, a novel vision-based tactile sensor with an ultra-compact cross-sectional diameter of 8mm, designed for seamless integration into mainstream RAMIS devices, particularly the Da Vinci surgical systems. MiniTac features a novel mechanoresponsive photonic elastomer membrane that changes color distribution under varying contact pressures. This color change is captured by an embedded miniature camera, allowing MiniTac to detect tumors both on the tissue surface and in deeper layers typically obscured from endoscopic view. MiniTac's efficacy has been rigorously tested on both phantoms and ex-vivo tissues. By leveraging advanced mechanoresponsive photonic materials, MiniTac represents a significant advancement in integrating tactile sensing into RAMIS, potentially expanding its applicability to a wider array of clinical scenarios that currently rely on traditional surgical approaches.

Problem

• Overcoming lack of tactile sensing in robot-assisted surgery
• Detecting surface and subsurface tumors during minimally invasive procedures
• Integrating compact vision-based tactile sensors into surgical systems

Innovation

• Ultra-compact 8mm vision-based tactile sensor
• Mechanoresponsive photonic elastomer membrane technology
• Embedded miniature camera detects subsurface tumors