Consolidator: Learning Persistent Routed Memory Across Context Boundaries

2026-08-12

AI Summary

This work addresses the challenge of effectively preserving and leveraging memory states across context boundaries. It proposes a learnable memory consolidation mechanism implemented via a lightweight Consolidator module, which dynamically transforms and accumulates routed short-term memory into long-term memory without erasing existing long-term content. The consolidated long-term memory serves dual roles: as a retrievable knowledge store and as a guiding signal for selecting subsequent memory slots, thereby significantly enhancing cross-context memory retention. Built upon the Phasor memory network architecture and incorporating KV cache eviction with hierarchical routing, the approach trains only the Consolidator—comprising 12.35K parameters (0.041% of the total model)—while keeping the backbone frozen. On a two-phase modulo-10 mapping task, this method improves recall accuracy for updated mappings from 44.38% to 87.02%.

Abstract

Copying short-term memory (STM) into a slower store can preserve state across a context boundary, but persistence alone does not ensure that the retained state influences subsequent memory access. We test this distinction in a Phasor Memory Network (PMNet) using Consolidator, a shared slot-local operator that transforms routed STM before accumulating it into long-term memory (LTM), without replaying the source tokens. After each consolidation, the KV cache and STM are cleared. The retained LTM can still be read and is also fed into the hierarchical router, thereby conditioning which explicit-memory slots subsequent inputs access. We evaluate this mechanism on a two-segment modulo-10 mapping task in which the second segment updates the mapping at the same memory address. Following a second consolidation and reset, a held-out query must recover the updated mapping from LTM. The backbone and memory interface are frozen, leaving only 12.35K Consolidator parameters trainable (0.041\% of a 29.95M model). Across five paired runs from the same STM-pretraining checkpoint, direct LTM routing raises updated-mapping recall from $44.38\pm1.94\%$ to $87.02\pm1.76\%$ ($+42.64\pm1.10$ percentage points), while immediate STM recall remains 89.90\% in both conditions; both train separate Consolidators and retain the same LTM read paths. Learned consolidation outperforms forced identity accumulation by $21.40\pm1.91$ percentage points without routing and $68.70\pm1.76$ with routing. Thus, on this task, consolidated LTM serves as both retrievable content and an access state that shapes subsequent slot selection.

Problem

• long-term memory
• context boundary
• memory consolidation
• memory routing
• persistent memory

Innovation

• Consolidator
• Phasor Memory Network
• long-term memory
• memory routing
• context boundary