Personal Visual Memory from Explicit and Implicit Evidence

2026-05-27

AI Summary

Existing personalized AI memory systems rely excessively on textual inputs, overlooking the explicit and implicit user-specific information embedded in images. This work systematically distinguishes and models these two types of visual evidence for the first time, introducing the VisualMem architecture: a structured visual memory module that integrates multimodal context to infer user identity, affiliations, and persistent facts—going beyond merely converting images into generic textual descriptions. We present the first visual memory benchmark tailored for personalized AI agents and demonstrate that our approach significantly outperforms existing methods on this benchmark while maintaining competitive performance on standard text-based memory tasks. These results establish visual memory as an indispensable and distinct component of long-term personalized memory systems.

Abstract

Long-term memory is increasingly important for personalized AI agents, yet existing benchmarks and methods remain largely text-centric. Even when images are included, the user-specific information needed for later questions is typically recoverable from text alone, and most memory systems reduce image turns to generic captions. Yet images often carry personal information that text rarely states -- both explicit evidence, such as recurring user-associated entities, and implicit evidence, such as latent user facts inferred from visual or multimodal cues. We introduce a benchmark for personal visual memory that targets both forms of evidence, and propose VisualMem, a hybrid visual--text architecture that augments a text-memory backend with a structured personal visual memory module. Rather than collapsing images into captions, VisualMem uses conversational context to resolve identity, ownership, and durable user facts. Experiments show that VisualMem substantially outperforms prior memory systems on our benchmark while remaining competitive on standard text-memory benchmarks, indicating that personal visual memory is a distinct and important component of long-term memory for personalized AI agents.

Problem

• personal visual memory
• explicit evidence
• implicit evidence
• multimodal cues
• long-term memory

Innovation

• personal visual memory
• VisualMem
• multimodal memory
• implicit visual evidence
• hybrid visual-text architecture