Problem
Research questions and friction points this paper is trying to address.
Articulated Object Reconstruction
Language Grounding
Open-vocabulary Control
Semantic Kinematics
ArtLang: Structured Language-to-Kinematics Grounding for Articulated 3D Actuation
🤖 AI Summary
This study addresses the challenges of semantic deficiency and natural language control in articulated object reconstruction by proposing the ArtLang framework. The method constructs a semantic-motion articulation graph representation that integrates a typed parser with a global graph matching reasoning mechanism, enabling precise mapping of open-vocabulary instructions to joint targets while effectively supporting ambiguity rejection. Experiments on both synthetic and real-world datasets demonstrate that the framework achieves reliable semantic grounding and continuous control across repetitive components, spatial references, and ambiguous instructions. Consequently, this approach significantly enhances the linguistic interactivity and robustness of reconstructed assets, facilitating more effective natural language interaction with 3D articulated models in complex scenarios.
📝 Abstract
Articulated-object reconstructions recover explicit geometry and kinematics, but their parts often remain semantically anonymous and must be controlled through part indices and numerical joint parameters. We present ArtLang, a framework for open-vocabulary language control of persistent reconstructed articulated assets. ArtLang represents an asset as a semantic-kinematic articulation graph and augments its surface with language features and graph-constrained motion. Open-vocabulary proposals are bound to reconstructed parts while allowing uncertain parts to remain unnamed. A typed parser converts a command into a directive graph containing referring expressions, actions, magnitudes, reference frames, and relations. We then solve a global graph-to-graph grounding problem that jointly reasons about semantic, spatial, relational, and kinematic compatibility, with support for null assignments and abstention under ambiguity. Accepted directives are converted into continuous joint targets within the observed motion range and executed through forward kinematics. Experiments on synthetic reconstructions, mesh-based assets, and real captures demonstrate reliable language grounding and continuous articulated control across repeated parts, spatial references, relational commands, and ambiguous instructions.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Open-vocabulary language control
Semantic-kinematic articulation graph
Graph-to-graph grounding
Typed parser
Continuous articulated control
🔎 Similar Papers
No similar papers found.
