RGE-GS: Reward-Guided Expansive Driving Scene Reconstruction via Diffusion Priors

2025-06-28

AI Summary

Single driving sequences often yield incomplete road structure scans, hindering sensor simulators’ ability to regress realistic driving maneuvers. Method: We propose a diffusion-prior- and reward-guided 3D Gaussian splatting expansion framework for scene reconstruction. Specifically: (1) a diffusion model generates geometrically and semantically consistent scene priors; (2) a reward network filters physically stable generation modes; and (3) a scene-convergence-aware differential Gaussian optimization strategy enhances training efficiency and reconstruction stability. Results: Our method achieves state-of-the-art performance on public benchmarks, enabling high-fidelity, physically plausible extrapolative reconstruction of driving scenes. The source code is publicly available.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

A single-pass driving clip frequently results in incomplete scanning of the road structure, making reconstructed scene expanding a critical requirement for sensor simulators to effectively regress driving actions. Although contemporary 3D Gaussian Splatting (3DGS) techniques achieve remarkable reconstruction quality, their direct extension through the integration of diffusion priors often introduces cumulative physical inconsistencies and compromises training efficiency. To address these limitations, we present RGE-GS, a novel expansive reconstruction framework that synergizes diffusion-based generation with reward-guided Gaussian integration. The RGE-GS framework incorporates two key innovations: First, we propose a reward network that learns to identify and prioritize consistently generated patterns prior to reconstruction phases, thereby enabling selective retention of diffusion outputs for spatial stability. Second, during the reconstruction process, we devise a differentiated training strategy that automatically adjust Gaussian optimization progress according to scene converge metrics, which achieving better convergence than baseline methods. Extensive evaluations of publicly available datasets demonstrate that RGE-GS achieves state-of-the-art performance in reconstruction quality. Our source-code will be made publicly available at https://github.com/CN-ADLab/RGE-GS. (Camera-ready version incorporating reviewer suggestions will be updated soon.)

Problem

• Incomplete road structure scanning in single-pass driving clips
• Cumulative physical inconsistencies in diffusion prior integration
• Compromised training efficiency in 3D Gaussian Splatting extension

Innovation

• Reward network prioritizes consistent diffusion patterns
• Differentiated training adjusts Gaussian optimization progress
• Synergizes diffusion generation with reward-guided integration