HyReach: Vision-Guided Hybrid Manipulator Reaching in Unseen Cluttered Environments

2026-03-22

AI Summary

This work addresses the challenge of high-precision robotic grasping in unstructured, previously unseen cluttered environments, where conventional rigid manipulators struggle due to limited compliance and adaptability. The authors propose a real-time hybrid rigid-soft continuum manipulator system that integrates vision-guided 3D scene reconstruction, shape-aware motion planning, and a learning-based hybrid controller. This approach enables, for the first time, generalization to entirely novel scenes without requiring environment-specific retraining. By synergistically combining the compliance of soft robotics with the precision of rigid mechanisms, the system achieves an average end-effector positioning error of less than 2 cm across diverse real-world cluttered settings, substantially improving task success rates and robustness in open-world manipulation scenarios.

Technology Category

• Intelligent Robots: Manipulation
• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning
• Computer Vision: Vision for Robotics & Autonomous Driving

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

As robotic systems increasingly operate in unstructured, cluttered, and previously unseen environments, there is a growing need for manipulators that combine compliance, adaptability, and precise control. This work presents a real-time hybrid rigid-soft continuum manipulator system designed for robust open-world object reaching in such challenging environments. The system integrates vision-based perception and 3D scene reconstruction with shape-aware motion planning to generate safe trajectories. A learning-based controller drives the hybrid arm to arbitrary target poses, leveraging the flexibility of the soft segment while maintaining the precision of the rigid segment. The system operates without environment-specific retraining, enabling direct generalization to new scenes. Extensive real-world experiments demonstrate consistent reaching performance with errors below 2 cm across diverse cluttered setups, highlighting the potential of hybrid manipulators for adaptive and reliable operation in unstructured environments.

Problem

• hybrid manipulator
• unseen cluttered environments
• vision-guided reaching
• real-time manipulation
• open-world object reaching

Innovation

• hybrid manipulator
• vision-guided control
• continuum robotics
• shape-aware motion planning
• zero-shot generalization