DriveFix: Spatio-Temporally Coherent Driving Scene Restoration

2026-03-17

AI Summary

Existing methods for 4D driving scene reconstruction often suffer from spatial misalignment across cameras and temporal drift due to insufficient spatiotemporal consistency. This work proposes DriveFix, a novel framework that, for the first time, explicitly integrates spatiotemporal consistency constraints into a diffusion model. DriveFix employs an interleaved diffusion Transformer to jointly model cross-view spatial alignment and temporal dependencies, complemented by a geometry-aware loss and history-conditioned generation to ensure outputs adhere to a coherent 3D structure. Evaluated on Waymo, nuScenes, and PandaSet, the method achieves state-of-the-art performance in both scene reconstruction and novel view synthesis, significantly reducing visual artifacts and enhancing texture stability, thereby advancing high-fidelity 4D world modeling toward real-world deployment.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent advancements in 4D scene reconstruction, particularly those leveraging diffusion priors, have shown promise for novel view synthesis in autonomous driving. However, these methods often process frames independently or in a view-by-view manner, leading to a critical lack of spatio-temporal synergy. This results in spatial misalignment across cameras and temporal drift in sequences. We propose DriveFix, a novel multi-view restoration framework that ensures spatio-temporal coherence for driving scenes. Our approach employs an interleaved diffusion transformer architecture with specialized blocks to explicitly model both temporal dependencies and cross-camera spatial consistency. By conditioning the generation on historical context and integrating geometry-aware training losses, DriveFix enforces that the restored views adhere to a unified 3D geometry. This enables the consistent propagation of high-fidelity textures and significantly reduces artifacts. Extensive evaluations on the Waymo, nuScenes, and PandaSet datasets demonstrate that DriveFix achieves state-of-the-art performance in both reconstruction and novel view synthesis, marking a substantial step toward robust 4D world modeling for real-world deployment.

Problem

• spatio-temporal coherence
• 4D scene reconstruction
• autonomous driving
• multi-view synthesis
• temporal drift

Innovation

• spatio-temporal coherence
• diffusion transformer
• multi-view restoration
• 4D scene reconstruction
• geometry-aware training