StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance

2026-06-12

AI Summary

Current evaluations of agent memory are largely confined to isolated dialogues or task-specific improvements, failing to assess the full spectrum of memory capabilities—from continuous observation to future task assistance. This work proposes the first streaming memory evaluation paradigm oriented toward future assistance, constructing a two-stage task sequence based on first-person EgoLife videos to separately measure an agent’s ability to leverage observed evidence and reuse interactive feedback. We introduce four metrics to systematically evaluate memory performance across evidence recall, initial utilization, feedback integration, and subsequent reuse. Evaluations of eight memory systems on two backbone models reveal that even when evidence is stored or feedback is locally integrated, current methods struggle to effectively support downstream tasks, highlighting a critical bottleneck in future-oriented memory-assisted reasoning.

Abstract

A central role of personal-agent memory is to turn stored information and prior interactions into future-oriented assistance. In daily use, useful cues come from what the agent observes and how the user interacts with the agent, and the agent must carry them forward from the current request to similar future tasks. Existing memory benchmarks usually test dialogue recall or task improvement in isolation, leaving the trajectory from streaming observations to later assistance largely untested. We introduce StreamMemBench, a streaming benchmark that constructs a two-step task sequence around each evidence anchor from EgoLife egocentric streams. The initial task tests evidence use, while the follow-up task tests whether feedback and interaction experience are reused. Four metrics diagnose evidence recall, initial evidence use, feedback incorporation, and follow-up reuse. Experiments with eight memory systems across two backbones show that current systems often fail to use observed evidence or turn feedback into reliable follow-up behavior, even when evidence is stored or feedback is incorporated locally. StreamMemBench is publicly available at https://github.com/landian60/StreamMemBench.

Problem

• agent memory
• streaming evaluation
• future-oriented assistance
• memory benchmark
• evidence reuse

Innovation

• streaming memory evaluation
• future-oriented assistance
• evidence reuse
• agent memory benchmark
• two-step task sequence