ContactFusion: Stochastic Poisson Surface Maps from Visual and Contact Sensing

2025-03-20

AI Summary

To address insufficient pose estimation robustness in tight-tolerance robotic assembly caused by point-cloud noise and contact uncertainty, this paper proposes a multimodal perception method integrating global geometry and local contact information. Our approach features three key contributions: (1) a novel force/torque inversion algorithm based on rejection sampling, enabling high-precision estimation of end-effector contact location; (2) embedding contact observations into the Stochastic Poisson Surface Reconstruction (SPSR) framework to construct an online-updatable Stochastic Poisson Surface Map (SPSMap) with explicit uncertainty modeling; and (3) tightly coupled fusion of point-cloud and six-axis force/torque sensor data. Simulation results demonstrate significant improvement in contact localization accuracy. Real-world peg-in-hole experiments show a 42% reduction in hole pose estimation error and a 76% decrease in insertion failure rate.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Location- and context-aware Web and WoT applications and services
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Robust and precise robotic assembly entails insertion of constituent components. Insertion success is hindered when noise in scene understanding exceeds tolerance limits, especially when fabricated with tight tolerances. In this work, we propose ContactFusion which combines global mapping with local contact information, fusing point clouds with force sensing. Our method entails a Rejection Sampling based contact occupancy sensing procedure which estimates contact locations on the end-effector from Force/Torque sensing at the wrist. We demonstrate how to fuse contact with visual information into a Stochastic Poisson Surface Map (SPSMap) - a map representation that can be updated with the Stochastic Poisson Surface Reconstruction (SPSR) algorithm. We first validate the contact occupancy sensor in simulation and show its ability to detect the contact location on the robot from force sensing information. Then, we evaluate our method in a peg-in-hole task, demonstrating an improvement in the hole pose estimate with the fusion of the contact information with the SPSMap.

Problem

• Robotic assembly precision hindered by noisy scene understanding
• Fusing visual and contact sensing for improved pose estimation
• Enhancing peg-in-hole tasks with stochastic surface mapping

Innovation

• Combines global mapping with local contact sensing
• Uses Rejection Sampling for contact occupancy estimation
• Fuses visual and contact data into Stochastic Poisson Map