SCOUT: Self-Checking and Recovery-Aware Tool-Thought Agents for Ultra-Long Egocentric Video Reasoning

2026-08-08

AI Summary

Understanding ultra-long first-person videos presents significant challenges, including sparse critical evidence, limited context window lengths, and unrecoverable errors from faulty reasoning. To address these issues, this work proposes SCOUT, a framework built upon a Chain-of-Tool-Thought architecture that incorporates a self-checking mechanism to dynamically evaluate tool invocation outcomes. SCOUT further introduces an uncertainty-prioritized strategy optimization method, UPS-GRPO, combined with turn-level advantage decomposition, enabling adaptive balancing between exploration and exploitation as well as efficient credit assignment. The proposed approach achieves state-of-the-art performance on ultra-long video benchmarks while maintaining competitive results on standard long-video tasks.

Abstract

Ultra-long egocentric video understanding requires reasoning over temporally sparse evidence distributed across hours or days, challenging current multimodal models with limited context and the grounding of key video segments. While Chain-of-Tool-Thought (CoTT) agent systems enable iterative retrieval and inspection, they suffer from error propagation due to rigid zoom-in strategies that lack recovery mechanisms. In this work, we address these challenges through SCOUT (Self-Checking Chain-Of-Tool-thought), a recovery-aware agentic framework introducing an adaptive policy that evaluates intermediate tool observations and dynamically trades off exploitation (zoom-in) and exploration (region switching), enabling robust multi-hop reasoning over extremely long horizons. However, training such multi-turn tool-using agents remains challenging, as existing RL methods rely on sparse outcome-level rewards and lack supervision over extended decision trajectories, resulting in suboptimal credit assignment for long-horizon reasoning. To address this, we develop UPS-GRPO, an uncertainty-prioritized policy optimization method that concentrates exploration on high-uncertainty post-tool states while preserving sample efficiency. We further introduce a turn-level advantage decomposition that integrates outcome rewards with tool-grounded temporal alignment rewards for improved credit assignment. Experiments show that SCOUT achieves state-of-the-art results on ultra-long egocentric benchmarks, while remaining competitive on shorter-horizon long-video settings.

Problem

• ultra-long egocentric video reasoning
• error propagation
• credit assignment
• recovery mechanism
• long-horizon reasoning

Innovation

• recovery-aware reasoning
• adaptive exploration-exploitation
• uncertainty-prioritized RL
• tool-grounded credit assignment
• ultra-long video understanding