RelAfford6D: Relational 6D Affordance Graphs for Constraint-Driven Robotic Manipulation

2026-06-25

AI Summary

This work addresses the disconnect between semantic abstraction and precise physical control in open-world robotic manipulation, as well as the lack of rigorous kinematic constraints for complex articulated objects. The authors propose a training-free, zero-shot manipulation framework that constructs a relational 6D affordance graph to parse natural language instructions into a semantic topological structure. Leveraging vision foundation models, this structure is lifted into SE(3) poses, formalizing the task as a kinematic constraint satisfaction problem to generate physically plausible, continuous trajectories. By unifying semantic part relationships with exact SE(3) geometric constraints for the first time, the method achieves robust, cross-category manipulation resilient to environmental perturbations. Experiments demonstrate significant improvements over existing data-driven approaches in both simulation and real-world settings, setting new benchmarks in zero-shot success rate, generalization, and execution robustness.

Abstract

Bridging abstract semantics and precise physical control remains a fundamental challenge in open-world robotic manipulation. While recent data-driven policies show promise, their reliance on isolated contact points or latent affordance embeddings lacks the rigorous kinematic constraints necessary for complex articulated objects.To overcome the limitation, we introduce RelAfford6D, a novel training-free framework centered on a Relational 6D Affordance Graph. Given a free-form instruction, our system deduces a semantic topology linking a primary interacting part to its physical anchor. By elevating these topological nodes into precise metric $SE(3)$ poses via vision foundation models, we analytically formulate downstream execution as a kinematic constraint satisfaction problem. The robot synthesizes continuous trajectories by tracking strictly defined physical manifolds (e.g., revolute or prismatic orbits). Coupled with a closed-loop tracking mechanism for dynamic replanning against disturbances, our physically grounded approach achieves superior zero-shot success rates, cross-category generalization and execution robustness in both simulation and the real world environments, outperforming existing data-driven baselines.

Problem

• robotic manipulation
• 6D affordance
• kinematic constraints
• articulated objects
• semantic grounding

Innovation

• Relational 6D Affordance Graph
• kinematic constraint satisfaction
• SE(3) pose estimation
• zero-shot manipulation
• physical manifolds