A Contact-Driven Framework for Manipulating in the Blind

2025-10-22

AI Summary

To address the challenges of distinguishing free from occupied space and predicting unknown structures under visually degraded conditions (e.g., occlusion, low illumination) in robotic manipulation, this paper proposes a blind manipulation framework integrating contact feedback with structural priors. Methodologically, we introduce structural priors into a contact-aware particle filter to generate extrapolatable local occupancy maps and design a noise-robust path planner, enabling closed-loop coordination among contact sensing, occupancy estimation, and motion planning. Technically, the framework fuses joint torque-based contact detection, a history-driven occupancy prediction network, and a noise-resilient planning algorithm. Evaluated on both simulation and physical UR10e platforms, our approach successfully completes under-sink valve operation and cluttered shelf retrieval tasks, reducing task completion time by up to 50%. Ablation studies confirm the effectiveness and necessity of each component.

Technology Category

• Intelligent Robots: Manipulation
• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Robots often face manipulation tasks in environments where vision is inadequate due to clutter, occlusions, or poor lighting--for example, reaching a shutoff valve at the back of a sink cabinet or locating a light switch above a crowded shelf. In such settings, robots, much like humans, must rely on contact feedback to distinguish free from occupied space and navigate around obstacles. Many of these environments often exhibit strong structural priors--for instance, pipes often span across sink cabinets--that can be exploited to anticipate unseen structure and avoid unnecessary collisions. We present a theoretically complete and empirically efficient framework for manipulation in the blind that integrates contact feedback with structural priors to enable robust operation in unknown environments. The framework comprises three tightly coupled components: (i) a contact detection and localization module that utilizes joint torque sensing with a contact particle filter to detect and localize contacts, (ii) an occupancy estimation module that uses the history of contact observations to build a partial occupancy map of the workspace and extrapolate it into unexplored regions with learned predictors, and (iii) a planning module that accounts for the fact that contact localization estimates and occupancy predictions can be noisy, computing paths that avoid collisions and complete tasks efficiently without eliminating feasible solutions. We evaluate the system in simulation and in the real world on a UR10e manipulator across two domestic tasks--(i) manipulating a valve under a kitchen sink surrounded by pipes and (ii) retrieving a target object from a cluttered shelf. Results show that the framework reliably solves these tasks, achieving up to a 2x reduction in task completion time compared to baselines, with ablations confirming the contribution of each module.

Problem

• Developing a contact-driven framework for blind robot manipulation in cluttered environments
• Integrating contact feedback with structural priors to navigate unseen obstacles
• Enabling robust task completion without vision using localization and planning

Innovation

• Contact-driven manipulation using joint torque sensing
• Occupancy mapping with learned predictors for unexplored areas
• Robust planning accounting for noisy contact localization