PRISM3D: Probabilistic Refinement and Robust Initialization for Physically Consistent Scene Modeling under Extreme Motion Blur

2026-07-04

AI Summary

This work addresses the failure of conventional Structure-from-Motion (SfM) methods under extreme motion blur by proposing PRISM3D, a unified framework that, for the first time, leverages deep dense tracking for robust initialization of 3D Gaussian splatting. Integrating a physics-based imaging model grounded in Bézier trajectories with a Markov Chain Monte Carlo (MCMC)-based probabilistic densification strategy, PRISM3D enables physically consistent blind 3D reconstruction. Furthermore, the authors introduce PRISM3D-E, the first multimodal reconstruction framework fusing RGB and event camera data, along with a dedicated benchmark dataset. Experiments demonstrate that the proposed approach significantly outperforms existing methods under extreme blur conditions, with both its single-modality and multimodal variants achieving state-of-the-art performance.

Abstract

We address the inverse problem of blind 3D scene reconstruction from extremely motion-blurred images, a scenario where traditional Structure-from-Motion (SfM) pipelines fail. Existing approaches typically circumvent this bottleneck by relying on impractical sharp-image supervision. In this work, we introduce PRISM3D, a unified framework enabling robust reconstruction directly from severely degraded inputs. To overcome the lack of a reliable starting point, we propose a Robust Initialization strategy utilizing deep dense tracking method (VGGSfM) to recover global topology where feature matching fails. To the best of our knowledge, we are the first to effectively leverage this paradigm to bootstrap 3D Gaussian Splatting from extreme motion blur. However, while robust, this initialization yields sparse and noisy geometry that causes deterministic optimization to diverge. To resolve this, we propose a coupled solution driven by probability and physics: we adopt a probabilistic formulation for geometric densification via Markov Chain Monte Carlo (MCMC) to robustly populate the sparse priors, while simultaneously modeling physical image formation via continuous Bezier Trajectories. Furthermore, while PRISM3D establishes a highly robust standalone pipeline, the availability of complementary event streams offers an opportunity to push the reconstruction fidelity further. To exploit this, we introduce PRISM3D-E, a multi-modal (RGB + Events) extension that seamlessly integrates high-temporal-resolution events as structural priors to maximize geometric recovery. Because existing datasets lack paired event streams under such severe degradation, we concurrently contribute the PRISM3D-E Benchmark to facilitate rigorous evaluation. Extensive experiments demonstrate that both our standalone RGB framework and its multi-modal extension establish new state-of-the-art performance.

Problem

• motion blur
• 3D scene reconstruction
• blind inverse problem
• Structure-from-Motion
• extreme degradation

Innovation

• motion blur
• 3D reconstruction
• probabilistic refinement
• robust initialization
• event-based vision