Towards Version-aware Operations and Transaction Memories for Multi-layer MeMo

2026-06-22

AI Summary

This work addresses the challenge of knowledge updating in large language models, which typically necessitates costly retraining. Building upon the Compositional Multi-layer Memory (CMM) architecture, the authors propose a version-aware operational layer that compiles high-level semantic edits into ordered, composable memory primitive transactions. By introducing versioned CMM and transactional CMM, knowledge modifications are modeled as reversible and reusable structured transactions, enabling fine-grained replacement, rollback, historical tracing, and localized updates. This approach substantially reduces reliance on full model retraining while ensuring editing efficiency and traceability.

Abstract

MeMo proposes language models with explicit multi-layer correlation matrix memories (CMMs), where memorization, retrieval, and forgetting are architectural operations. This paper asks how such memories can reduce the need for retraining when knowledge changes. For changes expressible as MeMo memory associations, the model's accessible knowledge can be updated by editing explicit memories rather than retraining the whole model. We propose a version-aware operation layer in which high-level operations such as replace, obsolete, keep-history, rollback, and trace are compiled into MeMo-native primitive calls over sequences and tokens. The key observation is that a version-aware operation is rarely a single MeMo association. It is an ordered transaction of primitive edits, for example forgetting one sequence-token chain, memorizing another, preserving a historical chain, and recording an inverse program. The framework introduces two auxiliary CMMs: a Version CMM (V-CMM) for mapping version transitions to transaction handles, and a Transaction CMM (T-CMM) for storing reusable change contents and inverse programs. It supports both direct sequence-level edits and structured diff-level inputs, and outlines an evaluation route for update success, rollback, traceability, locality, and transaction reuse.

Problem

• version-aware operations
• transaction memories
• multi-layer MeMo
• knowledge updating
• memory editing

Innovation

• version-aware operations
• transaction memories
• correlation matrix memories
• memory editing
• model update without retraining