LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation

2026-08-28

AI Summary

为解决视频生成中长期一致性问题，提出LayerRecall方法，通过状态条件记忆路由选择性地在特定层注入历史信息，并利用跨时域预测匹配进行监督。

Abstract

Autoregressive video diffusion enables scalable long-video generation by producing chunks from a bounded recent context. While recency-based caching preserves local continuity, it evicts historical cues needed when subjects, objects, scenes, or attributes reappear. Existing memory mechanisms expose models to nonlocal history, but access alone does not ensure effective use. Our analysis reveals that video DiT layers exhibit distinct preferences for current, recent, and distant context, suggesting that long-range memory requires deciding both what to retrieve and where to use it. We introduce LayerRecall, a current-conditioned, layer-selective memory router that retrieves relevant historical K/V states and injects them only into backbone-specific memory-sensitive layers while preserving local attention elsewhere. To reduce reliance on scarce high-quality long-horizon videos and explicit memory-allocation labels, we further propose Cross-Horizon Prediction Matching (CHPM), which uses a privileged long-context reference to supervise the bounded-memory router in prediction space. Across 100 multi-shot evaluation prompts, LayerRecall achieves the best overall results on MemoBench and MovieBench while matching its backbone on VBench-Long, demonstrating stronger long-range recovery without sacrificing local continuity. Qualitative analyses further reveal memory-guided self-correction, whereby initially mismatched local attributes return to their historical appearance without resetting ongoing motion or scene structure. Additional analyses show cross-backbone portability and negligible inference overhead.

Problem

• video generation
• long-horizon consistency
• memory mechanisms
• historical context
• autoregressive video diffusion

Innovation

• LayerRecall
• State-Conditioned Memory Router
• Cross-Horizon Prediction Matching (CHPM)
• Long-Horizon Consistency
• Video Generation