Incremental Transformer Neural Processes

2026-02-21

AI Summary

This work addresses the inefficiency of existing Transformer Neural Processes (TNPs) in handling continuous data streams, which require recomputation of all internal representations and thus hinder efficient incremental updates. The authors propose the incremental Transformer Neural Process (incTNP), which, for the first time, integrates causal masking and Key-Value caching into neural processes, combined with an autoregressive training strategy. This approach enables linear-time updates upon new observations while preserving implicit Bayesian consistency. Empirical evaluations on tasks such as tabular regression and temperature forecasting demonstrate that incTNP achieves performance comparable to or better than non-causal TNPs, while delivering orders-of-magnitude speedup in inference.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: Generation
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance

Abstract

Neural Processes (NPs), and specifically Transformer Neural Processes (TNPs), have demonstrated remarkable performance across tasks ranging from spatiotemporal forecasting to tabular data modelling. However, many of these applications are inherently sequential, involving continuous data streams such as real-time sensor readings or database updates. In such settings, models should support cheap, incremental updates rather than recomputing internal representations from scratch for every new observation -- a capability existing TNP variants lack. Drawing inspiration from Large Language Models, we introduce the Incremental TNP (incTNP). By leveraging causal masking, Key-Value (KV) caching, and a data-efficient autoregressive training strategy, incTNP matches the predictive performance of standard TNPs while reducing the computational cost of updates from quadratic to linear time complexity. We empirically evaluate our model on a range of synthetic and real-world tasks, including tabular regression and temperature prediction. Our results show that, surprisingly, incTNP delivers performance comparable to -- or better than -- non-causal TNPs while unlocking orders-of-magnitude speedups for sequential inference. Finally, we assess the consistency of the model's updates -- by adapting a metric of ``implicit Bayesianness", we show that incTNP retains a prediction rule as implicitly Bayesian as standard non-causal TNPs, demonstrating that incTNP achieves the computational benefits of causal masking without sacrificing the consistency required for streaming inference.

Problem

• Incremental Learning
• Transformer Neural Processes
• Streaming Data
• Sequential Inference
• Neural Processes

Innovation

• Incremental Learning
• Transformer Neural Processes
• Causal Masking
• KV Caching
• Implicit Bayesianness