MegaMem: A Retrieval Solution for Ultra-Large Context Windows

2026-08-22

AI Summary

为解决超大上下文窗口的检索问题，MegaMem采用双视图检索系统，通过分离语义访问与生成证据来提高检索准确性和效率。

Abstract

Modern language models and agents increasingly require persistent memory for complete codebases, long interaction histories, and heterogeneous enterprise records. The key challenge is to keep hundreds of millions of tokens searchable while passing only bounded source evidence to the answer model. We introduce MegaMem, a source-resolved dual-view retrieval system that separates semantic access from generation evidence. Distilled records and detailed evidence are searched with original and transformed queries; every distilled hit resolves to an immutable source ID before reciprocal-rank fusion, deduplication, and cross-encoder reranking; and only the highest-ranked detailed evidence within a fixed budget supports generation. Post-answer attribution then identifies which loaded sources support the fixed answer. We evaluate MegaMem on EnterpriseRAG-Bench, which contains more than 500,000 heterogeneous enterprise documents and approximately 650M tokens. MegaMem improves Overall from 68.22 to 82.26 and reaches 86.50 Correctness. These results show that MegaMem supports ultra-large persistent memory while preserving strong answer accuracy under a bounded generation context. By separating searchable memory scale from answer-context size, MegaMem provides a practical path toward accurate retrieval over memories ranging from hundreds of millions to one billion tokens. Our code is available at https://github.com/ xfab-xinyuansong/MegaMem.git.

Problem

• persistent memory
• ultra-large context windows
• retrieval system
• searchable tokens
• answer model

Innovation

• Dual-View Retrieval
• Persistent Memory
• Ultra-Large Context Windows
• Reciprocal-Rank Fusion
• Cross-Encoder Reranking