CAIRN: Cross-Room 3D Scene Understanding with Topology-Aware Large Multimodal Models

2026-07-07

AI Summary

Existing 3D large language models struggle to reason about cross-room object relationships and spatial topology in realistic multi-room indoor environments. To address this limitation, this work proposes a topology-aware multi-room 3D-LLM that explicitly models inter-room structure by introducing learnable room-level tokens and a topology-guided hierarchical attention mechanism, effectively integrating room-level abstractions with object-level relations to enable cross-room reasoning. The model further enhances object context through graph neural networks, incorporates geometric-biased attention, and leverages multi-task instruction fine-tuning. Evaluated on the newly introduced CAIRN-MR multi-room benchmark, the proposed approach significantly outperforms existing methods while maintaining competitive performance across five established single-room benchmarks.

Abstract

Existing 3D scene-grounded Large Language Models (3D-LLMs) focus on answering questions grounded in simplified single-room 3D scenes, lacking the ability to reason over real-world household environments containing multiple interconnected rooms and diverse object categories. We introduce CAIRN, a topology-aware 3D-LLM for multi-room 3D scene understanding. CAIRN aligns transformer attention with scene hierarchy, giving the model explicit awareness of object-level relations and room-level connectivity. It enriches object tokens with room-local relational context via a graph neural network, introduces learned room tokens for room-level abstraction, and applies a hierarchical attention mask with geometric bias to route information according to scene topology. CAIRN is developed on CAIRN-MR, a benchmark we introduce on HM3D for multi-room 3D scene understanding, covering grounding, captioning, and four question-answering tasks that progressively evaluate from intra-room perception to cross-room reasoning. Experiments show that CAIRN outperforms prior 3D-LLMs by a large margin across all CAIRN-MR tasks while remaining competitive on five single-room benchmarks.

Problem

• 3D scene understanding
• multi-room reasoning
• Large Language Models
• scene topology
• cross-room perception

Innovation

• topology-aware
• multi-room 3D scene understanding
• hierarchical attention
• graph neural network
• 3D large language model