EntroPE: Entropy-Guided Dynamic Patch Encoder for Time Series Forecasting

2025-09-30

AI Summary

Existing Transformer-based time series forecasting models employ fixed-length patching, which disrupts temporal continuity and impairs modeling of short-term dependencies. To address this, we propose EntroPE—the first entropy-guided, conditionally adaptive patching framework for time series encoding. Leveraging conditional entropy as an information-theoretic criterion, EntroPE automatically identifies intrinsic structural transition points in the sequence, enabling semantics-preserving, variable-length segmentation. Combined with localized pooling and global cross-attention Transformers, EntroPE jointly captures short-range dynamics and long-range patterns. By breaking the rigid fixed-patch paradigm, EntroPE establishes a new modeling paradigm for time series. Extensive experiments on multiple long-horizon forecasting benchmarks demonstrate significant improvements: average MAE reduction of 12.3% and FLOPs reduction of 18.7%, confirming both enhanced prediction accuracy and computational efficiency.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Transformer-based models have significantly advanced time series forecasting, with patch-based input strategies offering efficiency and improved long-horizon modeling. Yet, existing approaches rely on temporally-agnostic patch construction, where arbitrary starting positions and fixed lengths fracture temporal coherence by splitting natural transitions across boundaries. This naive segmentation often disrupts short-term dependencies and weakens representation learning. In response, we propose EntroPE (Entropy-Guided Dynamic Patch Encoder), a novel, temporally informed framework that dynamically detects transition points via conditional entropy and dynamically places patch boundaries. This preserves temporal structure while retaining the computational benefits of patching. EntroPE consists of two key modules, namely an Entropy-based Dynamic Patcher (EDP) that applies information-theoretic criteria to locate natural temporal shifts and determine patch boundaries, and an Adaptive Patch Encoder (APE) that employs pooling and cross-attention to capture intra-patch dependencies and produce fixed-size latent representations. These embeddings are then processed by a global transformer to model inter-patch dynamics. Experiments across long-term forecasting benchmarks demonstrate that EntroPE improves both accuracy and efficiency, establishing entropy-guided dynamic patching as a promising new paradigm for time series modeling. Code is available at: https://github.com/Sachithx/EntroPE.

Problem

• Dynamically detects transition points via conditional entropy
• Preserves temporal structure in patch-based time series forecasting
• Improves accuracy and efficiency in long-term forecasting benchmarks

Innovation

• Dynamic patch boundaries guided by conditional entropy
• Entropy-based detection of natural temporal transition points
• Adaptive patch encoding with pooling and cross-attention