MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models

2026-01-17

AI Summary

This work addresses the absence of systematic benchmarks for evaluating reward models in long-term memory management within large language models. To this end, the authors introduce the first benchmark specifically designed for assessing reward models in this context, featuring long-context understanding and generation tasks spanning 8K–128K tokens and encompassing ten distinct memory management strategies. The study conducts a comprehensive evaluation of thirteen state-of-the-art reward models, revealing that the performance gap between open-source and closed-source models has substantially narrowed, with newer generations consistently outperforming their predecessors—improvements not attributable to parameter count alone. Furthermore, the analysis uncovers fundamental limitations in current reward models’ ability to evaluate memory-related capabilities, thereby offering critical insights and directions for future research.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Existing works increasingly adopt memory-centric mechanisms to process long contexts in a segment manner, and effective memory management is one of the key capabilities that enables large language models to effectively propagate information across the entire sequence. Therefore, leveraging reward models (RMs) to automatically and reliably evaluate memory quality is critical. In this work, we introduce MemoryRewardBench, the first benchmark to systematically study the ability of RMs to evaluate long-term memory management processes. MemoryRewardBench covers both long-context comprehension and long-form generation tasks, featuring 10 distinct settings with different memory management patterns, with context length ranging from 8K to 128K tokens. Evaluations on 13 cutting-edge RMs indicate a diminishing performance gap between open-source and proprietary models, with newer-generation models consistently outperforming their predecessors regardless of parameter count. We further expose the capabilities and fundamental limitations of current RMs in evaluating LLM memory management across diverse settings.

Problem

• reward models
• long-term memory management
• large language models
• long-context comprehension
• benchmark

Innovation

• reward models
• long-term memory management
• benchmark
• large language models
• long-context evaluation