A3R: Agentic Affordance Reasoning via Cross-Dimensional Evidence in 3D Gaussian Scenes

2026-04-02

AI Summary

This work addresses the challenge that static, single-view observations are insufficient for fine-grained functional affordance reasoning in complex 3D Gaussian scenes. To overcome this limitation, the paper introduces the first agent-based, cross-dimensional evidence acquisition framework, reframing affordance reasoning as a sequential interaction process. The proposed approach employs an MLLM-driven policy to actively select actions within the 3D Gaussian scene, dynamically fusing 3D geometric and 2D semantic evidence. Sequential decision-making is optimized using the GRPO algorithm. Evaluated on multiple scene-level benchmarks, the method significantly outperforms static prediction models, demonstrating that dynamic, cross-dimensional evidence gathering effectively enhances affordance reasoning accuracy.

Abstract

Affordance reasoning in 3D Gaussian scenes aims to identify the region that supports the action specified by a given text instruction in complex environments. Existing methods typically cast this problem as one-shot prediction from static scene observations, assuming sufficient evidence is already available for reasoning. However, in complex 3D scenes, many failure cases arise not from weak prediction capacity, but from incomplete task-relevant evidence under fixed observations. To address this limitation, we reformulate fine-grained affordance reasoning as a sequential evidence acquisition process, where ambiguity is progressively reduced through complementary 3D geometric and 2D semantic evidence. Building on this formulation, we propose A3R, an agentic affordance reasoning framework that enables an MLLM-based policy to iteratively select evidence acquisition actions and update the affordance belief through cross-dimensional evidence acquisition. To optimize such sequential decision making, we further introduce a GRPO-based policy learning strategy that improves evidence acquisition efficiency and reasoning accuracy. Extensive experiments on scene-level benchmarks show that A3R consistently surpasses static one-shot baselines, demonstrating the advantage of agentic cross-dimensional evidence acquisition for fine-grained affordance reasoning in complex 3D Gaussian scenes.

Problem

• affordance reasoning
• 3D Gaussian scenes
• evidence acquisition
• cross-dimensional reasoning
• complex environments

Innovation

• Agentic Reasoning
• Cross-Dimensional Evidence
• 3D Gaussian Scenes
• Affordance Reasoning
• Sequential Decision Making