Falcon-X: A Time Series Foundation Model for Heterogeneous Multivariate Modeling

2026-05-26

AI Summary

Existing foundation models for time series struggle to effectively capture semantic alignment and complex interactions—such as synergistic or antagonistic relationships—among heterogeneous multivariate signals. To address this limitation, this work proposes a unified prototype space mapping that embeds variables into a shared latent space and introduces a differential attention mechanism to explicitly model both positive and negative semantic affinities, thereby overcoming the expressive constraints of conventional non-negative attention. Furthermore, a request-dispatch mechanism enables zero-shot structural transfer of variable trajectories. The proposed approach achieves state-of-the-art performance on the GIFT-Eval and fev-bench benchmarks, establishing a scalable new paradigm for modeling complex multivariate time series.

Abstract

Time series foundation models (TSFMs) are transforming the forecasting paradigm through large-scale cross-domain pretraining. However, most existing TSFMs remain univariate, and recent efforts to enable cross-variate modeling still operate directly within the raw variate space. This design introduces fundamental limitations in semantic alignment and relational expressivity. Specifically, raw-space group mixing lacks a dedicated mechanism to align heterogeneous physical quantities, while standard non-negative attention fails to capture the complex synergistic and antagonistic interactions ubiquitous in real-world systems. To address these challenges, we propose Falcon-X, decouples variates from the raw space and maps them into a unified latent prototype space. Falcon-X employs a Unified Prototype Diff-Attention mechanism that explicitly evaluates both positive and negative semantic affinities to explicitly align heterogeneous variates. Cross-variate interactions are then efficiently performed within this shared space via Latent Entity Attention, naturally facilitating zero-shot structural transfer. Finally, a Variate Reassembly Router robustly reconstructs variate-specific trajectories via a request-and-dispatch mechanism. Extensive evaluations on the GIFT-Eval and fev-bench benchmarks demonstrate that Falcon-X achieves state-of-the-art forecasting performance, offering a principled and scalable paradigm for complex multivariate environments. Falcon-X is publicly released to support future research.

Problem

• time series foundation models
• heterogeneous multivariate modeling
• semantic alignment
• cross-variate interactions
• relational expressivity

Innovation

• time series foundation model
• heterogeneous multivariate modeling
• latent prototype space
• Diff-Attention
• zero-shot structural transfer