NeuroVoxel-LM: Language-Aligned 3D Perception via Dynamic Voxelization and Meta-Embedding

2025-07-26

AI Summary

To address the slow feature extraction and coarse-grained semantic representation of 3D language models on large-scale sparse point clouds, this paper proposes a NeRF-enhanced Dynamic Resolution Multi-Scale Voxelization (DR-MSV) framework, integrated with a lightweight Token-Adaptive Pooling Lightweight Meta-Embedding (TAP-LME), attention-weighted fusion, and residual integration. The key contributions are: (i) DR-MSV jointly optimizes geometric fidelity and computational efficiency via adaptive voxel resolution across scales; (ii) TAP-LME enables token-level semantic adaptive pooling, significantly enhancing fine-grained semantic expressiveness. Evaluated on multiple 3D language understanding benchmarks, our method achieves a 2.1× speedup and a +4.7% mAP improvement over state-of-the-art approaches, substantially outperforming conventional voxelization and pooling strategies. This work establishes a new efficient and precise multimodal representation paradigm for language-driven 3D perception in large-scale scenes.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Recent breakthroughs in Visual Language Models (VLMs) and Multimodal Large Language Models (MLLMs) have significantly advanced 3D scene perception towards language-driven cognition. However, existing 3D language models struggle with sparse, large-scale point clouds due to slow feature extraction and limited representation accuracy. To address these challenges, we propose NeuroVoxel-LM, a novel framework that integrates Neural Radiance Fields (NeRF) with dynamic resolution voxelization and lightweight meta-embedding. Specifically, we introduce a Dynamic Resolution Multiscale Voxelization (DR-MSV) technique that adaptively adjusts voxel granularity based on geometric and structural complexity, reducing computational cost while preserving reconstruction fidelity. In addition, we propose the Token-level Adaptive Pooling for Lightweight Meta-Embedding (TAP-LME) mechanism, which enhances semantic representation through attention-based weighting and residual fusion. Experimental results demonstrate that DR-MSV significantly improves point cloud feature extraction efficiency and accuracy, while TAP-LME outperforms conventional max-pooling in capturing fine-grained semantics from NeRF weights.

Problem

• Improves 3D perception with dynamic voxelization for sparse point clouds
• Enhances feature extraction efficiency and accuracy in large-scale scenes
• Boosts semantic representation via lightweight meta-embedding and adaptive pooling

Innovation

• Dynamic resolution voxelization for efficient 3D processing
• Lightweight meta-embedding enhances semantic representation
• NeRF integration improves feature extraction accuracy