EAG3R: Event-Augmented 3D Geometry Estimation for Dynamic and Extreme-Lighting Scenes

2025-11-30

AI Summary

To address the poor robustness of monocular RGB-based 3D geometric estimation under dynamic object motion and extreme illumination, this paper proposes EAG3R—an event-augmented, pose-free reconstruction framework. Building upon MonST3R, EAG3R introduces a retina-inspired image enhancement module and an SNR-aware lightweight event adapter to construct spatiotemporally aligned RGB-event representations. It further designs an event-driven photometric consistency loss, enabling end-to-end global optimization without retraining. The key innovation lies in the first deep integration of asynchronous event streams with RGB depth estimation, leveraging SNR-aware fusion to enhance feature reliability in low-light and high-motion scenarios. Experiments demonstrate that EAG3R significantly outperforms RGB-only methods on monocular depth estimation, pose tracking, and dynamic scene reconstruction—while maintaining stable performance under extreme illumination without requiring nighttime-specific training data.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Robust 3D geometry estimation from videos is critical for applications such as autonomous navigation, SLAM, and 3D scene reconstruction. Recent methods like DUSt3R demonstrate that regressing dense pointmaps from image pairs enables accurate and efficient pose-free reconstruction. However, existing RGB-only approaches struggle under real-world conditions involving dynamic objects and extreme illumination, due to the inherent limitations of conventional cameras. In this paper, we propose EAG3R, a novel geometry estimation framework that augments pointmap-based reconstruction with asynchronous event streams. Built upon the MonST3R backbone, EAG3R introduces two key innovations: (1) a retinex-inspired image enhancement module and a lightweight event adapter with SNR-aware fusion mechanism that adaptively combines RGB and event features based on local reliability; and (2) a novel event-based photometric consistency loss that reinforces spatiotemporal coherence during global optimization. Our method enables robust geometry estimation in challenging dynamic low-light scenes without requiring retraining on night-time data. Extensive experiments demonstrate that EAG3R significantly outperforms state-of-the-art RGB-only baselines across monocular depth estimation, camera pose tracking, and dynamic reconstruction tasks.

Problem

• Robust 3D geometry estimation in dynamic and extreme-lighting scenes
• Overcoming limitations of RGB-only methods in challenging real-world conditions
• Enhancing reconstruction with event streams for better spatiotemporal coherence

Innovation

• Event-augmented pointmap reconstruction with adaptive RGB-event fusion
• Retinex-inspired enhancement and SNR-aware fusion for reliability
• Event-based photometric loss for spatiotemporal coherence in optimization