Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoning

2026-07-01

AI Summary

Existing approaches struggle to perform multi-hop narrative reasoning on hour-long videos under memory-constrained online settings. To address this challenge, this work proposes Homer, a model-agnostic framework that, for the first time, integrates explicit causal and temporal relationships into a hierarchical memory system aligned with the video’s multi-scale structure. Homer further introduces a verifiable and error-correcting multi-turn retrieval mechanism inspired by human memory recall, enabling causal-driven reasoning. The framework significantly enhances the long-video understanding capabilities of various multimodal large language models, achieving performance gains of 5.5, 10.8, and 4.4 points over previous state-of-the-art methods on M3-Bench-robot, M3-Bench-web, and Video-MME-Long, respectively.

Abstract

Multimodal large language models excel on short clips but struggle on hour-long videos in an online setting, where frames are processed incrementally under limited memory. Existing online methods either retain compact visual representations that lack semantic structure, or build higher-level memory stores organized around temporal proximity rather than explicit causal links, leaving multi-hop narrative reasoning to be reconstructed by the LLM at every query. We bridge this gap with \textsc{Homer}, a Hierarchical Online Memory Exploration and Reasoning framework. \textsc{Homer}'s memory mirrors the multi-scale structure of long videos, ranging from raw perception, to recurring entities, to events connected by explicit temporal and causal relations. Its agentic reasoner then explores this memory the way humans do, locating the relevant scene, looking up details, and composing the answer through multi-round memory retrieval, with a harness that verifies and corrects each step. \textsc{Homer} outperforms the previous best agent method by $+5.5$, $+10.8$, and $+4.4$ points on M3-Bench-robot, M3-Bench-web, and Video-MME-Long, and consistently lifts three various LLM backbones, indicating a model-agnostic structural capability for grounded retrieval over long videos.

Problem

• long-form videos
• online setting
• limited memory
• multi-hop narrative reasoning
• causal relations

Innovation

• Hierarchical Memory
• Agentic Reasoning
• Long-form Video Understanding
• Online Video Processing
• Causal Event Modeling