FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting

2026-08-12

AI Summary

This work addresses the limitations of existing large language model (LLM)-based time series forecasting approaches, which rely on computationally expensive textual prompting for modality alignment and overlook spectral dynamics. The authors propose a prompt-free, frequency-aware autoregressive framework that directly injects spectral information into a frozen LLM via a Fourier analysis network. To effectively disentangle periodic and aperiodic components, they design an asymmetric Mixture-of-Experts (MoE) decoder and introduce a joint time–frequency loss to enhance long-range stability. Evaluated across 11 benchmarks encompassing 78 metrics, the method achieves state-of-the-art performance on 59 metrics, reducing average MSE and MAE by 5.3% and 5.6%, respectively, compared to the strongest LLM baseline—with improvements reaching up to 8.4%. It also demonstrates robust performance in few-shot and zero-shot settings.

Abstract

Recent advances in Large Language Models (LLMs) have spurred cross-modal solutions for time-series forecasting. However, existing methods rely heavily on textual prompts for modality alignment-introducing nontrivial computational overhead and failing to leverage the rich spectral dynamics inherent in time-series data. To enable prompt-free, frequency-aware adaptation of frozen LLMs, we propose FM-LLM (Frequency-Enhanced Mixture-of-Experts for adapting LLMs to Time Series Forecasting), an autoregressive framework grounded in constrained asymmetric coupling. A Fourier Analysis Network (FAN)-based spectral token aligner injects structured harmonic representations directly into the frozen LLM with numerical compatibility. An asymmetric Mixture-of-Experts (MoE) decoder enforces role separation: shared experts with lightweight FAN layers reconstruct the global periodic backbone, while routed experts-restricted to standard FFNs-specialize in modeling non-periodic residual dynamics. A time-frequency hybrid loss function jointly optimizes temporal accuracy and spectral consistency, mitigating error accumulation during long-horizon autoregressive rollouts. Evaluated across eleven public benchmarks, FM-LLM achieves state-of-the-art performance on 59 out of 78 evaluation metrics. Compared to the strongest autoregressive LLM-based baseline, it delivers average improvements of 5.3% in MSE and 5.6% in MAE, with maximum gains reaching 8.0% for MSE and 8.4% for MAE. FM-LLM also demonstrates robust transferability, maintaining superior performance in 10% few-shot and zero-shot forecasting scenarios.

Problem

• time series forecasting
• Large Language Models
• frequency-aware adaptation
• modality alignment
• spectral dynamics

Innovation

• Frequency-enhanced
• Mixture-of-Experts
• Fourier Analysis Network
• Prompt-free adaptation
• Time-series forecasting