Multimodal Adversarial Quality Policy for Safe Grasping

2026-03-02

AI Summary

This work addresses the safety risks associated with relying solely on RGB modality and the limited adaptability of existing multimodal (RGB-D) approaches in vision-guided robotic grasping. To this end, the authors propose the Multimodal Adversarial Quality Policy (MAQP), which establishes the first safety-oriented RGB-D grasping framework. MAQP integrates a Heterogeneous Dual-Patch Optimization Scheme (HDPOS) and a Gradient-Level Modality Balancing Strategy (GLMBS), incorporating modality-specific initialization, gradient reweighting, and distance-adaptive perturbation bounds to effectively mitigate inter-modal distributional discrepancies and optimization imbalances. Experimental results demonstrate that MAQP significantly enhances the robustness and safety of multimodal grasping on both standard datasets and collaborative robotic platforms.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements

Abstract

Vision-guided robot grasping based on Deep Neural Networks (DNNs) generalizes well but poses safety risks in the Human-Robot Interaction (HRI). Recent works solved it by designing benign adversarial attacks and patches with RGB modality, yet depth-independent characteristics limit their effectiveness on RGBD modality. In this work, we propose the Multimodal Adversarial Quality Policy (MAQP) to realize multimodal safe grasping. Our framework introduces two key components. First, the Heterogeneous Dual-Patch Optimization Scheme (HDPOS) mitigates the distribution discrepancy between RGB and depth modalities in patch generation by adopting modality-specific initialization strategies, employing a Gaussian distribution for depth patches and a uniform distribution for RGB patches, while jointly optimizing both modalities under a unified objective function. Second, the Gradient-Level Modality Balancing Strategy (GLMBS) is designed to resolve the optimization imbalance from RGB and Depth patches in patch shape adaptation by reweighting gradient contributions based on per-channel sensitivity analysis and applying distance-adaptive perturbation bounds. We conduct extensive experiments on the benchmark datasets and a cobot, showing the effectiveness of MAQP.

Problem

• safe grasping
• multimodal adversarial attack
• RGBD modality
• human-robot interaction
• vision-guided robot grasping

Innovation

• Multimodal Adversarial Quality Policy
• Heterogeneous Dual-Patch Optimization
• Gradient-Level Modality Balancing
• Safe Grasping
• RGB-D Adversarial Attack