HDR-NSFF: High Dynamic Range Neural Scene Flow Fields

2026-03-09

AI Summary

This work addresses the limitations of traditional HDR methods, which rely on 2D pixel alignment and often suffer from ghosting and temporal inconsistencies in dynamic scenes. We propose the first approach that integrates HDR reconstruction with 4D dynamic scene representation, enabling high-fidelity HDR radiance field reconstruction from alternately exposed monocular video. Our method jointly models HDR radiance, 3D scene flow, geometry, and tone mapping through a hybrid framework combining neural radiance fields and 4D Gaussian splatting. To handle exposure-varying inputs, we leverage DINO features for exposure-invariant motion estimation and incorporate generative priors to compensate for the inherent ambiguities of monocular observation and information loss in overexposed regions. Evaluated on our newly introduced HDR-GoPro dataset, the proposed method achieves state-of-the-art performance, significantly improving radiance detail recovery and temporal consistency under complex exposure conditions.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Radiance of real-world scenes typically spans a much wider dynamic range than what standard cameras can capture. While conventional HDR methods merge alternating-exposure frames, these approaches are inherently constrained to 2D pixel-level alignment, often leading to ghosting artifacts and temporal inconsistency in dynamic scenes. To address these limitations, we present HDR-NSFF, a paradigm shift from 2D-based merging to 4D spatio-temporal modeling. Our framework reconstructs dynamic HDR radiance fields from alternating-exposure monocular videos by representing the scene as a continuous function of space and time, and is compatible with both neural radiance field and 4D Gaussian Splatting (4DGS) based dynamic representations. This unified end-to-end pipeline explicitly models HDR radiance, 3D scene flow, geometry, and tone-mapping, ensuring physical plausibility and global coherence. We further enhance robustness by (i) extending semantic-based optical flow with DINO features to achieve exposure-invariant motion estimation, and (ii) incorporating a generative prior as a regularizer to compensate for limited observation in monocular captures and saturation-induced information loss. To evaluate HDR space-time view synthesis, we present the first real-world HDR-GoPro dataset specifically designed for dynamic HDR scenes. Experiments demonstrate that HDR-NSFF recovers fine radiance details and coherent dynamics even under challenging exposure variations, thereby achieving state-of-the-art performance in novel space-time view synthesis. Project page: https://shin-dong-yeon.github.io/HDR-NSFF/

Problem

• High Dynamic Range
• Dynamic Scenes
• Ghosting Artifacts
• Temporal Inconsistency
• Monocular Video

Innovation

• HDR radiance fields
• 4D spatio-temporal modeling
• scene flow
• exposure-invariant motion estimation
• generative prior