XPG-RL: Reinforcement Learning with Explainable Priority Guidance for Efficiency-Boosted Mechanical Search

2025-04-29

AI Summary

To address the challenges of severe occlusion, partial observability, long-horizon planning, and robust state estimation in mechanical search (MS) tasks within cluttered environments, this paper proposes an interpretable, priority-guided reinforcement learning framework operating directly on raw RGB-D inputs. The method employs multimodal perception fusion and a hierarchical RL architecture. Its key contributions are: (1) a task-driven action priority modeling mechanism that enables dynamic switching of action primitives under joint semantic-geometric scene representation; and (2) a learnable adaptive threshold policy network that integrates domain knowledge with data-driven decision-making to enhance robustness in discrete action selection. Extensive evaluation in simulation and on real robotic platforms demonstrates significant improvements in task success rate and motion efficiency, achieving up to 4.5× higher long-horizon task efficiency compared to baseline methods.

Technology Category

• Search and Optimization: Learning to Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations

Abstract

Mechanical search (MS) in cluttered environments remains a significant challenge for autonomous manipulators, requiring long-horizon planning and robust state estimation under occlusions and partial observability. In this work, we introduce XPG-RL, a reinforcement learning framework that enables agents to efficiently perform MS tasks through explainable, priority-guided decision-making based on raw sensory inputs. XPG-RL integrates a task-driven action prioritization mechanism with a learned context-aware switching strategy that dynamically selects from a discrete set of action primitives such as target grasping, occlusion removal, and viewpoint adjustment. Within this strategy, a policy is optimized to output adaptive threshold values that govern the discrete selection among action primitives. The perception module fuses RGB-D inputs with semantic and geometric features to produce a structured scene representation for downstream decision-making. Extensive experiments in both simulation and real-world settings demonstrate that XPG-RL consistently outperforms baseline methods in task success rates and motion efficiency, achieving up to 4.5$ imes$ higher efficiency in long-horizon tasks. These results underscore the benefits of integrating domain knowledge with learnable decision-making policies for robust and efficient robotic manipulation.

Problem

• Efficient mechanical search in cluttered environments using reinforcement learning
• Explainable priority guidance for long-horizon planning under occlusion
• Dynamic action selection for target grasping and occlusion removal

Innovation

• Explainable priority-guided RL for mechanical search
• Context-aware switching among action primitives
• RGB-D fused with semantic and geometric features