ERF-GS: Reconstructing Fast Motion from Disjoint Event-RGB Viewpoints

2026-08-09

AI Summary

This work addresses the challenges faced by conventional video-based dynamic 3D reconstruction methods in high-speed motion scenarios, including low frame rates, severe motion blur, and misalignment between RGB and event camera viewpoints. The authors propose ERF-GS, a novel framework that, for the first time, integrates event camera data into both the joint optimization and densification stages of 3D Gaussian Splatting (3DGS), leveraging the high temporal resolution of events to enhance reconstruction quality. Through simulation training designed to closely mimic real-world conditions, ERF-GS enables event-driven learning without requiring RGB input and supports natural scene reconstruction under viewpoint separation between RGB and event streams. Experiments demonstrate that ERF-GS significantly outperforms baseline methods 4DGS and E-D3DGS on the Neu3D and NVIDIA datasets, effectively mitigating motion blur and viewpoint misalignment while substantially improving reconstruction accuracy in high-speed dynamic scenes.

Abstract

Deep learning-driven representations such as neural radiance fields (NeRFs) and 3D Gaussian splatting (3DGS) have revolutionized the field of dynamic 3D scene reconstruction with improved visual precision and scalability. However, the reconstruction of fast-moving objects remains a challenge; existing methods based on conventional frame-based videos often struggle in scenarios such as sports events and animal videography. We propose an event-RGB fusion Gaussian splatting (ERF-GS) framework that integrates event information into both optimization and densification stages of the Gaussian splatting pipeline, taking advantage of novel event sensors with high frame-rate. Unlike many other event-assisted scene reconstruction methods, ERF-GS was developed using realistic simulation settings and realizes event-based learning detached from RGB inputs. This design enables its application beyond straightforward synthetic data into the realm of natural video with complex layout, low frame rates and severe motion blur. Our experiments show that ERF-GS outperforms both the 4DGS baseline and the concurrent E-D3DGS on different variants of the Neu3D and Nvidia datasets which include blurry RGB frames and disjoint RGB-event viewpoints. Our code is available at https://github.com/andrewbxy/ERF-GS.

Problem

• fast motion reconstruction
• dynamic 3D scene reconstruction
• event-RGB fusion
• motion blur
• disjoint viewpoints

Innovation

• event camera
• 3D Gaussian splatting
• dynamic scene reconstruction
• fast motion
• RGB-event fusion