Accurate Parameter-Efficient Test-Time Adaptation for Time Series Forecasting

2025-06-29

AI Summary

Real-world time series often exhibit non-stationarity, causing significant performance degradation of pre-trained forecasting models during testing. To address this, we propose a parameter-efficient test-time adaptation (TTA) method that updates only lightweight input/output calibration modules—bypassing full-model fine-tuning. Our approach features: (1) low-rank adapters jointly optimized with dynamic gating mechanisms to model time-varying distributional shifts; and (2) a composite loss function integrating robustness regularization, frequency-domain periodic consistency, and patch-wise structural alignment. The method enables online, zero-shot calibration without retraining, introducing less than 0.1% additional parameters. Extensive experiments demonstrate state-of-the-art or competitive performance across multiple benchmark datasets, while maintaining full compatibility with diverse mainstream forecasting backbones.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Real-world time series often exhibit a non-stationary nature, degrading the performance of pre-trained forecasting models. Test-Time Adaptation (TTA) addresses this by adjusting models during inference, but existing methods typically update the full model, increasing memory and compute costs. We propose PETSA, a parameter-efficient method that adapts forecasters at test time by only updating small calibration modules on the input and output. PETSA uses low-rank adapters and dynamic gating to adjust representations without retraining. To maintain accuracy despite limited adaptation capacity, we introduce a specialized loss combining three components: (1) a robust term, (2) a frequency-domain term to preserve periodicity, and (3) a patch-wise structural term for structural alignment. PETSA improves the adaptability of various forecasting backbones while requiring fewer parameters than baselines. Experimental results on benchmark datasets show that PETSA achieves competitive or better performance across all horizons. Our code is available at: https://github.com/BorealisAI/PETSA

Problem

• Adapts forecasting models for non-stationary time series efficiently
• Reduces memory and compute costs via small calibration modules
• Maintains accuracy with specialized multi-component loss function

Innovation

• Parameter-efficient test-time adaptation with small modules
• Low-rank adapters and dynamic gating for representation adjustment
• Specialized loss combining robust, frequency, and structural terms