AERMANI-PLACE: Language Guided Object Placement with Aerial Manipulators

2026-06-12

AI Summary

Existing aerial manipulation systems rely on precise coordinate specifications for object placement, which lacks intuitiveness and complicates deployment. This work proposes a novel approach that integrates natural language instructions with image editing models: by generating scene images augmented with visual markers and leveraging depth observations to map these markers into physical space, the system guides an aerial manipulator to perform placement tasks. To the best of our knowledge, this is the first method to combine language guidance with image editing for intuitive human–robot interaction without requiring explicit coordinate inputs. The integrated pipeline encompasses image editing, depth-aware perception, visual localization, and trajectory generation. In evaluations across 100 language-guided tasks, the system achieves a success rate of 87% in simulation and 72% on a real-world platform.

Abstract

Object placement is a fundamental component of aerial manipulation tasks, yet existing systems typically require the desired placement position to be specified explicitly in metric coordinates. Such interfaces are not intuitive and require users to reason about coordinate frames and scene geometry, making them difficult to use in practical deployments. In contrast, humans often communicate spatial goals through a combination of language and pointing gestures. Inspired by this observation, we present AERMANI-PLACE, a framework for language-guided object placement with aerial manipulators. Given a scene image and a natural language instruction, an image editing model generates a modified version of the scene containing a visual marker that indicates where the object should be placed. This marker is then grounded into the physical environment using depth observations to recover a metric place point, after which a placement trajectory is generated and executed by the aerial manipulator. We evaluate the proposed approach on a test set of 100 language-guided placement tasks and demonstrate successful execution on a real aerial manipulation platform. Experimental results show that the proposed method reliably infers placement locations from language instructions with an average success rate of 87\% on the test-set and transfers effectively to real-world aerial manipulation with an average success rate of 72\%. Video: https://youtu.be/SgwwgLBsv0g

Problem

• object placement
• aerial manipulation
• language instruction
• human-robot interaction
• spatial reasoning

Innovation

• language-guided manipulation
• aerial manipulators
• object placement
• vision-language grounding
• image editing for robotics