A segmented robot grasping perception neural network for edge AI

2025-07-18

AI Summary

To address the challenge of achieving real-time, reliable, and energy-efficient robotic grasping perception on resource-constrained edge devices, this paper proposes a heatmap-guided 6-DoF grasp pose detection framework. Our method innovatively integrates hardware-aware optimization techniques—including input dimensionality reduction, model partitioning, and quantization—to enable end-to-end, fully on-chip inference on the GAP9 RISC-V microcontroller. To the best of our knowledge, this is the first work to support real-time grasp detection on the GraspNet-1Billion dataset using an ultra-low-power MCU-class chip, achieving inference latency under 30 ms and power consumption below 150 mW. The proposed framework significantly extends the applicability boundary of miniature embedded platforms for autonomous grasping tasks and establishes a deployable paradigm for edge-AI-driven lightweight robotic manipulation.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Hardware-aware ML
• Computer Vision: Biometrics, Face, Gesture & Pose

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Robotic grasping, the ability of robots to reliably secure and manipulate objects of varying shapes, sizes and orientations, is a complex task that requires precise perception and control. Deep neural networks have shown remarkable success in grasp synthesis by learning rich and abstract representations of objects. When deployed at the edge, these models can enable low-latency, low-power inference, making real-time grasping feasible in resource-constrained environments. This work implements Heatmap-Guided Grasp Detection, an end-to-end framework for the detection of 6-Dof grasp poses, on the GAP9 RISC-V System-on-Chip. The model is optimised using hardware-aware techniques, including input dimensionality reduction, model partitioning, and quantisation. Experimental evaluation on the GraspNet-1Billion benchmark validates the feasibility of fully on-chip inference, highlighting the potential of low-power MCUs for real-time, autonomous manipulation.

Problem

• Enables real-time robotic grasping in resource-constrained edge environments
• Optimizes neural network for low-power 6-Dof grasp pose detection
• Validates on-chip inference feasibility for autonomous manipulation on MCUs

Innovation

• Heatmap-Guided Grasp Detection for 6-Dof poses
• Hardware-aware optimization with model partitioning
• Fully on-chip inference on GAP9 RISC-V SoC