SoLAR: Error-Resilient Streamable Long-Horizon Free-Viewpoint Video Reconstruction with Anchor Activation and Latent Recalibration

2026-05-08

AI Summary

Existing free-viewpoint video methods suffer from degraded reconstruction quality and insufficient transmission robustness in long-duration scenes. This work proposes the first streaming-capable, GOP-free framework for long-term free-viewpoint video reconstruction, introducing Anchor Activation Dynamics (AAD) to model non-rigid transformations and a Latent Discrepancy Aware Recalibration (LaDAR) mechanism to suppress error propagation. By integrating rate-distortion-optimized dynamic anchor-based volumetric representations, the method achieves state-of-the-art reconstruction performance with minimal storage overhead, while maintaining real-time capability and practical deployability. This advancement facilitates the application of long-duration free-viewpoint video in immersive systems.

Abstract

Free-Viewpoint Video (FVV) has emerged as a cornerstone of next-generation immersive media systems and attracted widespread attention. Previous methods primarily focus on short video sequences and suffer from significant performance degradation when processing long-horizon free-viewpoint video (LFVV). Motivated by bit allocation theory, we analyze dynamic-anchor-based volumetric video representation within a rate-distortion optimization framework and propose \textbf{SoLAR}, which is the first error-resilient streamable FVV framework that maintains stable reconstruction quality on long sequences without requiring group-of-pictures partitioning. We propose the Anchor Activation Dynamics (AAD), which enables dynamic anchors to model non-rigid transformations by dynamically activating informative anchors and suppressing redundant ones. Furthermore, we introduce Latent Discrepancy Aware Recalibration (LaDAR), which is a mechanism to identify discrepancies between latent representations and recalibrate the correspondences encoded in the network, effectively mitigating error propagation in LFVV without compromising real-time performance or storage compactness. Extensive experiments demonstrate that \textbf{SoLAR} achieves state-of-the-art reconstruction performance while maintaining minimum storage overhead, which provides a new direction for LFVV reconstruction and advances the practical deployment of immersive systems. Demo free-viewpoint videos are provided in the supplementary material.

Problem

• Free-Viewpoint Video
• Long-Horizon Video
• Error Resilience
• Streamable Reconstruction
• Error Propagation

Innovation

• SoLAR
• Free-Viewpoint Video
• Error Resilience
• Anchor Activation
• Latent Recalibration