ScanBot: Towards Intelligent Surface Scanning in Embodied Robotic Systems

2025-05-22

AI Summary

Current vision-language-action (VLA) models fail to meet the sub-millimeter path continuity and parameter stability requirements of industrial laser scanning tasks. To address this, we introduce the first instruction-driven, high-precision surface scanning robot dataset, comprising 12 object categories and 6 industrial scanning tasks, with synchronized RGB, depth, laser profile, and robot pose data. We propose the first end-to-end framework jointly modeling natural language instructions and sub-millimeter motion constraints, integrating multimodal large language models (MLLMs), VLA modeling, motion planning, and multi-sensor temporal synchronization. Experiments expose systematic failures of existing VLA models in fine-grained instruction following under real-world precision constraints. Furthermore, we establish the first comprehensive high-precision scanning benchmark covering the full perception–planning–execution pipeline, enabling rigorous evaluation of robotic accuracy, robustness, and instruction fidelity in industrial surface inspection scenarios.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Language and Vision
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We introduce ScanBot, a novel dataset designed for instruction-conditioned, high-precision surface scanning in robotic systems. In contrast to existing robot learning datasets that focus on coarse tasks such as grasping, navigation, or dialogue, ScanBot targets the high-precision demands of industrial laser scanning, where sub-millimeter path continuity and parameter stability are critical. The dataset covers laser scanning trajectories executed by a robot across 12 diverse objects and 6 task types, including full-surface scans, geometry-focused regions, spatially referenced parts, functionally relevant structures, defect inspection, and comparative analysis. Each scan is guided by natural language instructions and paired with synchronized RGB, depth, and laser profiles, as well as robot pose and joint states. Despite recent progress, existing vision-language action (VLA) models still fail to generate stable scanning trajectories under fine-grained instructions and real-world precision demands. To investigate this limitation, we benchmark a range of multimodal large language models (MLLMs) across the full perception-planning-execution loop, revealing persistent challenges in instruction-following under realistic constraints.

Problem

• Addresses high-precision robotic surface scanning demands
• Overcomes limitations in instruction-following for fine-grained tasks
• Benchmarks MLLMs for industrial laser scanning accuracy

Innovation

• High-precision laser scanning dataset
• Instruction-conditioned robot trajectories
• Multimodal benchmark for VLA models