Beyond Distribution Shifts: Adaptive Hyperspectral Image Classification at Test Time

2025-09-10

AI Summary

Hyperspectral image classification models suffer from poor robustness under realistic degradations—including noise, blur, compression artifacts, and atmospheric effects. To address this, we propose HyperTTA, a unified test-time adaptation (TTA) framework for hyperspectral imagery. We first construct the first multi-degradation hyperspectral dataset and design a spectral-spatial Transformer classifier (SSTC) with a multi-scale receptive field mechanism. Our core contribution is the Confidence-aware Entropy-minimizing LayerNorm Adapter (CELA), which dynamically adapts model parameters solely by updating affine weights in LayerNorm layers—requiring neither source data nor labels. CELA is further enhanced by label smoothing and lightweight TTA to minimize prediction entropy for high-confidence samples. Extensive experiments on two benchmark hyperspectral datasets demonstrate that HyperTTA consistently outperforms state-of-the-art methods, achieving stable accuracy gains and strong cross-degradation generalization. The code will be publicly released.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Hyperspectral image (HSI) classification models are highly sensitive to distribution shifts caused by various real-world degradations such as noise, blur, compression, and atmospheric effects. To address this challenge, we propose HyperTTA, a unified framework designed to enhance model robustness under diverse degradation conditions. Specifically, we first construct a multi-degradation hyperspectral dataset that systematically simulates nine representative types of degradations, providing a comprehensive benchmark for robust classification evaluation. Based on this, we design a spectral-spatial transformer classifier (SSTC) enhanced with a multi-level receptive field mechanism and label smoothing regularization to jointly capture multi-scale spatial context and improve generalization. Furthermore, HyperTTA incorporates a lightweight test-time adaptation (TTA) strategy, the confidence-aware entropy-minimized LayerNorm adapter (CELA), which updates only the affine parameters of LayerNorm layers by minimizing prediction entropy on high-confidence unlabeled target samples. This confidence-aware adaptation prevents unreliable updates from noisy predictions, enabling robust and dynamic adaptation without access to source data or target annotations. Extensive experiments on two benchmark datasets demonstrate that HyperTTA outperforms existing baselines across a wide range of degradation scenarios, validating the effectiveness of both its classification backbone and the proposed TTA scheme. Code will be made available publicly.

Problem

• Enhancing HSI classification robustness against distribution shifts
• Adapting models to real-world degradations without source data
• Addressing noise, blur, compression, and atmospheric effects dynamically

Innovation

• Multi-degradation hyperspectral dataset construction
• Spectral-spatial transformer with multi-level receptive field
• Confidence-aware entropy-minimized LayerNorm adapter