FUSE: Active Functional Affordance Grounding through Adaptive Semantic-Geometric Evidence Acquisition

2026-08-12

AI Summary

This work addresses the challenge faced by embodied agents in fixed-view settings, where occlusion or incompleteness of functional cues impedes reliable object localization. To tackle this issue, the paper introduces the task of active functional affordance localization and proposes FUSE—a framework that integrates uncertainty-driven active exploration with a learned amortized planner to adaptively gather semantic-geometric evidence. The approach guides the agent to select observation viewpoints maximizing information gain. Key contributions include the formalization of this novel task, the design of an adaptive observation strategy that unifies explicit exploration with amortized planning, and the creation of a new benchmark built upon Habitat. Experiments demonstrate that FUSE achieves state-of-the-art localization performance without prior knowledge, reduces computational overhead by 1.33× compared to fully explicit exploration, and remains compatible with diverse affordance knowledge sources.

Abstract

Embodied agents must often identify and interact with objects based on their function rather than their identity, requiring them to actively acquire observations that reveal discriminative functional evidence. Existing affordance grounding methods operate from fixed viewpoints and lack mechanisms for deciding where to look when functional cues are occluded or incomplete. We introduce Active Functional Affordance Grounding, a new task in which an agent sequentially explores a scene to identify and spatially ground an object satisfying a functional query. To address this problem, we propose FUSE, an adaptive semantic-geometric evidence acquisition framework that combines explicit uncertainty-driven exploration with a learned amortized planner to efficiently select informative viewpoints. We further introduce a Habitat-based benchmark for evaluating active functional grounding. Experiments show that FUSE achieves the highest observed non-oracle grounding performance while reducing computation by 1.33x relative to fully explicit exploration, and remains effective across multiple affordance knowledge sources.

Problem

• affordance grounding
• active exploration
• functional query
• viewpoint selection
• embodied agents

Innovation

• Active Exploration
• Functional Affordance Grounding
• Semantic-Geometric Evidence
• Amortized Planning
• Uncertainty-driven Viewpoint Selection