Bounded Context Management for Tabular Foundation Models on Stream Learning

2026-06-17

AI Summary

This work addresses performance degradation in tabular stream learning caused by distribution shifts by proposing CURE, a model-agnostic strategy that eschews parameter updates and instead focuses on context management. Drawing on an analysis from the perspective of future information, it formalizes context management into three principles: retaining recent samples, preserving samples with high predictive uncertainty, and removing redundant instances. CURE dynamically maintains a bounded context window through an entropy-driven admission mechanism coupled with a redundancy-aware eviction policy. Evaluated across seven streaming datasets, CURE achieves up to a 27.0% improvement over conventional methods, demonstrating consistent and significant gains across diverse tabular foundation models and outperforming alternative context management strategies.

Abstract

Tabular stream learning requires predictions on sequentially arriving examples under distribution shift. While standard methods adapt by updating model states, tabular foundation models (TFMs) make predictions conditioned on a labeled context in an in-context manner, making them a natural alternative for stream learning. This shifts the challenge from how to update the model to how to manage the context. We propose a future information view that yields three practical requirements for context management: preserve recent examples, retain uncertain examples, and remove redundant examples. We instantiate these requirements as CURE (Context management via Uncertainty-aware admission and Redundancy aware Eviction), a context-managing policy with entropy-gated admission and redundancy-aware eviction. Across seven streams, CURE shows up to 27.0% relative improvement over classical stream learners, remains robust across multiple TFM backbones, and ranks first among other policy variants. Code and datasets are available at https://github.com/morcellinus/CURE-ICML-FMSD.

Problem

• tabular stream learning
• distribution shift
• context management
• tabular foundation models
• bounded context

Innovation

• tabular foundation models
• stream learning
• context management
• uncertainty-aware admission
• redundancy-aware eviction