SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMs

2026-02-26

AI Summary

Existing 3D large vision-language models (LVLMs) employ RoPE positional encoding, which struggles to effectively model three-dimensional spatial structures and neglects angular dependencies, thereby limiting the model’s sensitivity to directional variations. To address this limitation, this work proposes Spherical Positional Encoding (SoPE), which introduces the spherical coordinate system into 3D LVLMs for the first time. SoPE unifies the representation of radial distance and azimuthal angles for point cloud tokens and incorporates a multi-scale frequency mixing mechanism to enhance both consistency and richness in geometric representation. Experimental results demonstrate that SoPE significantly improves performance across multiple 3D scene understanding benchmarks, exhibiting strong generalization capabilities.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

3D Large Vision-Language Models (3D LVLMs) built upon Large Language Models (LLMs) have achieved remarkable progress across various multimodal tasks. However, their inherited position-dependent modeling mechanism, Rotary Position Embedding (RoPE), remains suboptimal for 3D multimodal understanding. The vanilla RoPE formulation fails to preserve essential three-dimensional spatial structures when encoding 3D tokens, and its relative distance computation overlooks angular dependencies, hindering the model's ability to capture directional variations in visual representations. To overcome these limitations, we introduce Spherical Coordinate-based Positional Embedding (SoPE). Our method maps point-cloud token indices into a 3D spherical coordinate space, enabling unified modeling of spatial locations and directional angles. This formulation preserves the inherent geometric structure of point-cloud data, enhances spatial awareness, and yields more consistent and expressive geometric representations for multimodal learning. In addition, we introduce a multi-scale frequency mixing strategy to fuse feature information across different frequency domains. Experimental results on multiple 3D scene benchmarks validate the effectiveness of our approach, while real-world deployment experiments further demonstrate its strong generalization capability.

Problem

• 3D Large Vision-Language Models
• Rotary Position Embedding
• spatial perception
• spherical coordinates
• point-cloud representation

Innovation

• Spherical Coordinate
• Positional Embedding
• 3D LVLMs
• Spatial Perception
• Frequency Mixing