TriggerBench: Investigating Prospective Memory for Large Language Models

2026-06-22

AI Summary

This study addresses the neglect of prospective memory—the ability to spontaneously recall and act upon implicit constraints without explicit prompts—in current large language model (LLM) evaluations, which predominantly focus on retrospective memory. The authors introduce TriggerBench, the first systematic benchmark for prospective memory, spanning five dimensions of everyday assistance and professional workflows. Through retrospective memory controls, positive–negative sample comparisons, and overloaded trigger designs, the benchmark uniformly assesses models’ proactive recall accuracy, false-positive rates, and attentional robustness. Integrating scenario-based evaluation, controlled experiments, context-scaling analysis, and joint tracing with AIME-2025 reasoning trajectories, the work reveals a precision–recall trade-off in prospective memory, markedly inferior performance compared to retrospective memory, and sharp degradation with increasing context length. Notably, prospective memory accuracy is higher in successful reasoning trajectories, suggesting its utility as a behavioral probe of residual reasoning capacity.

Abstract

While Large Language Models (LLMs) are increasingly deployed in long interactions, existing evaluations focus predominantly on retrospective memory (RM) via explicit queries. Prospective memory (PM), the critical ability to spontaneously recall and act on latent constraints without direct prompts, remains largely unevaluated. We introduce TriggerBench, a comprehensive PM benchmark spanning five dimensions across both daily assistants and professional workflows. TriggerBench pairs scenarios with matched RM controls, contrastive positive/negative variants, and overloaded triggers, enabling fine-grained measurement of proactive recall, false-alarm rate, and attentional robustness under a single protocol. Our evaluation yields three key findings. (i) PM shows a precision-recall trade-off and attentional fragility. Though enhanced reasoning significantly improves proactive recall, models may overfit to an "always-remind" heuristic. Furthermore, PM accuracy degrades substantially under implicit constraints or triggers overloaded by concurrent user requests, indicating that robust PM remains an open challenge. (ii) PM is notably harder than RM: on identical contexts, RM near-saturates up to 100K tokens, while PM decays sharply as context length scales. (iii) PM may serve as a behavioral probe of spare reasoning capacity. Pairing PM scenarios with AIME-2025 math problems reveals that successful trajectories yield higher PM accuracy than failed ones at the same context length, showing PM tracks spare reasoning budget that token count obscures. Project page: https://github.com/KristenZHANG/TriggerBench-Official.

Problem

• prospective memory
• large language models
• memory evaluation
• implicit constraints
• long-context interaction

Innovation

• prospective memory
• large language models
• TriggerBench
• reasoning capacity
• memory benchmark