CapMem: A Benchmark for Caption-Based Episodic Memory in Egocentric Video

2026-09-15

AI Summary

研究通过引入CapMem基准，使用文本描述作为可重用的情景记忆来解决第一人称视频中情景记忆问题，方法包括CaptionQA和检索验证。

Abstract

Wearable assistants require episodic memory over egocentric video, yet current vision-language models face bounded frame budgets, growing visual-token costs, and long-context retrieval failures. Under these practical constraints, we study whether textual captions can serve as reusable episodic memory. We define the Episodic Memory Video Caption QA task and introduce CapMem, a human-annotated benchmark with 75 videos totaling 33.7 hours, and 1,000 multiple-choice questions across 16 scenarios. On long videos (>20 min), full-coverage CaptionQA with 30s and 60s caption windows outperforms direct VideoQA for 10/12 and 8/12 models, respectively. On the same video subset, a matched-frame control across six Qwen models retains mean accuracy gains of 3.22 and 2.55 points, respectively. Our caption-guided retrieve-and-verify harness further improves accuracy by up to 5.3 points. These results support the effectiveness of caption memory for episodic reasoning over long egocentric video.

Problem

• episodic memory
• egocentric video
• vision-language models
• frame budget
• visual-token costs

Innovation

• Episodic Memory
• Video Caption QA
• Caption-based Memory
• Egocentric Video
• Retrieval Efficiency