DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injection

2026-06-25

AI Summary

This work addresses the lack of interactive benchmarks for evaluating whether multimodal agents genuinely leverage visual memory, as existing research predominantly relies on textual representations. We propose DMV-Bench, the first evaluation benchmark specifically designed to assess visual memory in multimodal agents, built upon images of one thousand household items. By embedding incidental visual cues and requiring agents to retrieve specific items after long task sequences—while rigorously preventing textual leakage—the benchmark isolates pure visual memory capabilities. Inspired by dual-coding theory, we introduce DualMem, an asymmetric dual-encoding architecture that, for the first time, enables end-to-end visual-channel memory retention with language-channel query assistance. Experiments on models such as Gemini 2.5 Flash and Qwen2.5-VL-7B demonstrate that DualMem significantly outperforms caption-based baselines and existing multimodal memory systems across sequence lengths from 5 to 50, with consistent gains even after controlling for memory bank size and positional bias.

Abstract

Research on agent memory has matured rapidly, but almost entirely on the text side: few existing benchmarks ask, in an interactive environment, when an agent genuinely needs to remember what it saw rather than what it could write down. We introduce DMV-Bench (Code: https://github.com/yyyujintang/DMV-Bench), the first interactive benchmark for multimodal-agent visual memory. DMV-Bench is built on a controlled home-furnishing e-commerce catalogue of 1,000 product variants in which a text-leakage contract keeps the discriminative signal of each task in the pixels alone. Across a chain of autonomous shopping sessions, every visited product image carries a unique, pre-rendered incidental cue, and the agent is later asked to recall a particular cued product and navigate to its URL. Inspired by dual-coding theory, we propose DualMem, a memory architecture that maintains a visual and a verbal code in parallel. On DMV-Bench, DualMem outperforms a caption baseline and three recent multimodal agent-memory systems at every chain length J in {5, 10, 15, 50} on both Gemini 2.5 Flash and Qwen2.5-VL-7B, with the lead surviving controls for memory-bank size and encoding-position bias, and an asymmetric dual-coding regime in which vision carries the cue end-to-end while the verbal channel plays a smaller query-grounding role.

Problem

• visual memory
• multimodal agents
• long-horizon reasoning
• interactive benchmark
• incidental cue

Innovation

• visual memory
• multimodal agents
• incidental cue injection
• dual-coding theory
• interactive benchmark