Streaming Hierarchical Inference with Tabular Foundation Models

2026-09-07

AI Summary

为解决高吞吐量数据流中部署TFM的通信开销和延迟问题，提出HINT框架，结合边缘检索与云端TFM推理，平衡预测性能与通信成本。

Abstract

Tabular Foundation Models (TFMs) have recently demonstrated strong predictive performance through in-context learning, but their deployment in high-throughput data streams remains challenging due to communication overhead and latency. We propose \textit{HINT}, a hierarchical inference framework that combines edge-based retrieval with cloud-based TFM inference. A graph-based approximate nearest neighbor memory maintained over a sliding window provides local predictions and uncertainty estimates, allowing confident samples to be processed locally while uncertain instances are selectively offloaded, together with their retrieved context, to a cloud-hosted TFM. The framework exposes an offloading threshold and a neighborhood retrieval policy that can be varied to balance predictive performance and communication cost. Experiments show \textit{HINT} consistently identifies favorable trade-offs.

Problem

• Tabular Foundation Models
• high-throughput data streams
• communication overhead
• latency

Innovation

• Hierarchical Inference
• Tabular Foundation Models
• Edge-based Retrieval
• Cloud-based Inference
• Communication Cost