Exploring Active Learning for Label-Efficient Training of Semantic Neural Radiance Field

2025-07-23

AI Summary

Semantic Neural Radiance Fields (Semantic NeRFs) suffer from prohibitive reliance on costly pixel-level semantic annotations, severely hindering practical deployment. To address this, we propose a geometry-aware active learning framework tailored for Semantic NeRFs. Our core innovation lies in incorporating 3D geometric cues—such as surface normals, depth consistency, and voxel visibility—into uncertainty estimation, thereby prioritizing annotation efforts toward geometrically complex and semantically ambiguous regions. The framework enables end-to-end joint training of the query policy and NeRF optimization, supporting flexible annotation granularities (point-, patch-, or volume-level). Experiments on ScanNet and Replica demonstrate that our method achieves comparable mIoU to full supervision using only 50% of the annotation budget—yielding over 2.1× improvement in annotation efficiency and substantially lowering the barrier to Semantic NeRF deployment.

Technology Category

• Machine Learning: Active Learning
• Search and Optimization: Learning to Search
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Representation, semantic annotation, enhancement, enrichments, access and/or integration of a variety of data on the Web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Neural Radiance Field (NeRF) models are implicit neural scene representation methods that offer unprecedented capabilities in novel view synthesis. Semantically-aware NeRFs not only capture the shape and radiance of a scene, but also encode semantic information of the scene. The training of semantically-aware NeRFs typically requires pixel-level class labels, which can be prohibitively expensive to collect. In this work, we explore active learning as a potential solution to alleviate the annotation burden. We investigate various design choices for active learning of semantically-aware NeRF, including selection granularity and selection strategies. We further propose a novel active learning strategy that takes into account 3D geometric constraints in sample selection. Our experiments demonstrate that active learning can effectively reduce the annotation cost of training semantically-aware NeRF, achieving more than 2X reduction in annotation cost compared to random sampling.

Problem

• Reducing annotation cost for semantically-aware NeRF training
• Exploring active learning for label-efficient semantic NeRF
• Developing 3D-aware sample selection to minimize labeling effort

Innovation

• Active learning reduces semantic NeRF annotation cost
• 3D geometric constraints improve sample selection
• Achieves over 2X annotation cost reduction