Hestia: Hierarchical Next-Best-View Exploration for Systematic Intelligent Autonomous Data Collection

2025-08-01

AI Summary

To address the low efficiency of manual image acquisition in 3D reconstruction, this paper proposes a drone-based autonomous active exploration framework. Methodologically, the system formalizes the next-best-view (NBV) task by defining its observation space, action space (5-degree-of-freedom camera pose control), and reward function, and introduces a hierarchical reinforcement learning policy enabling multi-DOF viewpoint prediction and end-to-end deployment. The key contribution is the first integration of hierarchical NBV exploration with a real-world drone platform—trained in the NVIDIA IsaacLab simulation environment and successfully transferred to physical hardware. Extensive experiments across three benchmark datasets and cross-object generalization settings demonstrate robust performance: the approach achieves efficient, high-fidelity 3D reconstruction in both simulated and real-world scenarios, significantly improving automation in data acquisition and enhancing generalization capability.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Advances in 3D reconstruction and novel view synthesis have enabled efficient, photorealistic rendering, but the data collection process remains largely manual, making it time-consuming and labor-intensive. To address the challenges, this study introduces Hierarchical Next-Best-View Exploration for Systematic Intelligent Autonomous Data Collection (Hestia), which leverages reinforcement learning to learn a generalizable policy for 5-DoF next-best viewpoint prediction. Unlike prior approaches, Hestia systematically defines the next-best-view task by proposing core components such as dataset choice, observation design, action space, reward calculation, and learning schemes, forming a foundation for the planner. Hestia goes beyond prior next-best-view approaches and traditional capture systems through integration and validation in a real-world setup, where a drone serves as a mobile sensor for active scene exploration. Experimental results show that Hestia performs robustly across three datasets and translated object settings in the NVIDIA IsaacLab environment, and proves feasible for real-world deployment.

Problem

• Autonomous data collection is manual and inefficient
• Lack systematic next-best-view prediction for 3D reconstruction
• Existing methods lack real-world drone integration validation

Innovation

• Reinforcement learning for 5-DoF viewpoint prediction
• Systematic next-best-view task definition
• Real-world drone integration for scene exploration