When to Trust, How to Distill: Multi-Foundation Model Guidance for Lightweight, Robust Scientific Time Series Forecasting

2026-06-10

AI Summary

This work addresses the performance degradation and high computational cost of time series foundation models when applied zero-shot to scientific domains, which hinder their deployment on edge devices. To overcome these challenges, the authors propose GUARD, a framework that reformulates multi-teacher distillation as an instance-level decision process. GUARD employs context-aware routing to dynamically select the most relevant teacher model and introduces an uncertainty-aware gating mechanism with adaptive temperature scaling to modulate distillation intensity. This enables effective extraction of structured knowledge from distribution-mismatched foundation models to train lightweight, task-specific predictors. Evaluated across four scientific tasks—weather forecasting, carbon flux estimation, soil moisture prediction, and power grid monitoring—GUARD significantly reduces RMSE and outperforms the best-performing global foundation model on the most challenging 28.5% of samples, achieving both high accuracy and edge-deployment feasibility.

Abstract

The deployment of Time-Series Foundation Models (TSFMs) in physical sciences is hindered by a critical trade-off: while these models encode rich, universal temporal dynamics, they suffer from severe distributional misalignment when applied zero-shot to specific scientific domains, and their computational cost prohibits deployment in edge-computing sensor networks. We address a fundamental challenge: How can we extract latent structural knowledge from misaligned foundation models (FM) to train lightweight, specialized forecasters? We propose Gated Uncertainty-Aware Routing for Distillation (Guard), a novel framework that reframes multiteacher distillation as an instance-wise decision process with two adaptive mechanisms: (1) a Contextual Router that dynamically selects the most relevant teacher based on local input statistics, exploiting complementarity across diverse foundation models; and (2) an Uncertainty-Gated Temperature mechanism that acts as a "circuit-breaker," automatically attenuating distillation strength when teacher confidence diverges from domain reality. We evaluate our proposed lightweight framework on four climate-critical domains: meteorology, ecosystem carbon flux, soil moisture, and energy grids. Our method significantly reduces RMSE relative to a fixed-weight multi-teacher distillation baseline, successfully distilling knowledge from pretrained FMs (teachers) even when they exhibit suboptimal zero-shot accuracy due to distribution shift between the original and target data domains. We demonstrate that these domain-misaligned teachers can still serve as critical correctives, outperforming the globally superior FMs on 28.5% of the hardest instances. Ultimately, this enables high-precision scientific forecasting suitable for resource-constrained edge deployment. Code is available at https://github.com/RupasreeDey/GUARD-KDD2026.

Problem

• Time-Series Foundation Models
• Distributional Misalignment
• Knowledge Distillation
• Lightweight Forecasting
• Scientific Time Series

Innovation

• Time-Series Foundation Models
• Knowledge Distillation
• Uncertainty-Aware Routing
• Lightweight Forecasting
• Distribution Shift