NeuGrasp: Generalizable Neural Surface Reconstruction with Background Priors for Material-Agnostic Object Grasp Detection

2025-03-05

AI Summary

To address the challenge of robust grasping of transparent and specular objects against complex backgrounds, this paper proposes a material-agnostic neural surface reconstruction framework. The method jointly reconstructs object geometry and semantics from multi-view RGB inputs without requiring object-specific priors or material annotations. Key contributions include: (1) the first use of background prior volumetric representations to guide foreground object reconstruction; (2) a residual feature enhancement module coupled with occupancy-aware prior volume optimization, significantly improving generalization under narrow-baseline and sparse-view conditions; and (3) integration of Transformer-based multi-view feature aggregation, spatial encoding, and joint global–occupancy voxel modeling. Evaluated on both synthetic and real-world benchmarks, the framework achieves state-of-the-art grasping success rates while maintaining competitive surface reconstruction accuracy.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Robotic grasping in scenes with transparent and specular objects presents great challenges for methods relying on accurate depth information. In this paper, we introduce NeuGrasp, a neural surface reconstruction method that leverages background priors for material-agnostic grasp detection. NeuGrasp integrates transformers and global prior volumes to aggregate multi-view features with spatial encoding, enabling robust surface reconstruction in narrow and sparse viewing conditions. By focusing on foreground objects through residual feature enhancement and refining spatial perception with an occupancy-prior volume, NeuGrasp excels in handling objects with transparent and specular surfaces. Extensive experiments in both simulated and real-world scenarios show that NeuGrasp outperforms state-of-the-art methods in grasping while maintaining comparable reconstruction quality. More details are available at https://neugrasp.github.io/.

Problem

• Challenges in robotic grasping of transparent and specular objects
• Need for material-gnostic grasp detection methods
• Robust surface reconstruction in sparse viewing conditions

Innovation

• Leverages background priors for material-agnostic grasp detection
• Integrates transformers and global prior volumes for robust reconstruction
• Uses residual feature enhancement and occupancy-prior volume for spatial perception