🤖 AI Summary
In video prediction, RNN-based models suffer from progressive degradation of appearance details due to long-term memory accumulation, leading to significant quality deterioration as prediction horizon increases. To address this, we propose the Strongly Retrospective Video Prediction (SRVP) model, which introduces a novel dual-attention mechanism—integrating Standard Attention (SA) and Reinforced Feature Attention (RFA)—to explicitly decouple and jointly model spatiotemporal dependencies, thereby overcoming RNNs’ inherent detail-forgetting bottleneck. SRVP employs differentiable scaled dot-product attention for spatiotemporal feature fusion and enables end-to-end learning of high-fidelity spatiotemporal representations. Evaluated on three standard benchmarks, SRVP substantially mitigates quality degradation, achieving average improvements of +2.1 dB in PSNR and +0.035 in SSIM over strong RNN baselines, while matching the prediction accuracy of state-of-the-art RNN-free approaches.
📝 Abstract
Video prediction (VP) generates future frames by leveraging spatial representations and temporal context from past frames. Traditional recurrent neural network (RNN)-based models enhance memory cell structures to capture spatiotemporal states over extended durations but suffer from gradual loss of object appearance details. To address this issue, we propose the strong recollection VP (SRVP) model, which integrates standard attention (SA) and reinforced feature attention (RFA) modules. Both modules employ scaled dot-product attention to extract temporal context and spatial correlations, which are then fused to enhance spatiotemporal representations. Experiments on three benchmark datasets demonstrate that SRVP mitigates image quality degradation in RNN-based models while achieving predictive performance comparable to RNN-free architectures.