Bayesian Modelling of Multi-Year Crop Type Classification Using Deep Neural Networks and Hidden Markov Models

2025-10-08

AI Summary

Crop classification from multi-year satellite image time series (SITS) often suffers from temporal inconsistency and fails to capture genuine land-cover dynamics. Method: We propose a Bayesian joint modeling framework integrating a Transformer encoder for spectral-temporal feature extraction with a hidden Markov model (HMM) that explicitly encodes inter-class state transitions and observation uncertainty within a probabilistic framework, enabling coherent inference of long-term crop evolution patterns. Contribution/Results: To our knowledge, this is the first work embedding an HMM into a Transformer-based cascade architecture. Evaluated on a large-scale, six-year Sentinel-2 dataset covering 47 crop classes, our method significantly improves overall accuracy and F1-score, particularly enhancing temporal coherence across years. It establishes a novel, interpretable, and robust paradigm for dynamic land-cover monitoring.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Multi-modal Vision
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

The temporal consistency of yearly land-cover maps is of great importance to model the evolution and change of the land cover over the years. In this paper, we focus the attention on a novel approach to classification of yearly satellite image time series (SITS) that combines deep learning with Bayesian modelling, using Hidden Markov Models (HMMs) integrated with Transformer Encoder (TE) based DNNs. The proposed approach aims to capture both i) intricate temporal correlations in yearly SITS and ii) specific patterns in multiyear crop type sequences. It leverages the cascade classification of an HMM layer built on top of the TE, discerning consistent yearly crop-type sequences. Validation on a multiyear crop type classification dataset spanning 47 crop types and six years of Sentinel-2 acquisitions demonstrates the importance of modelling temporal consistency in the predicted labels. HMMs enhance the overall performance and F1 scores, emphasising the effectiveness of the proposed approach.

Problem

• Modeling temporal consistency in yearly land-cover map evolution
• Classifying multi-year crop types using satellite image time series
• Enhancing classification with Bayesian HMMs and deep neural networks

Innovation

• Combining deep learning with Bayesian modeling
• Using Hidden Markov Models with Transformer Encoder DNNs
• Capturing temporal correlations and crop patterns