MBTI: A Multi-Branch Efficient Fine-Tuning Framework for Hyperspectral Image Classification with Foundation Models

2026-07-14

AI Summary

This work addresses the challenge of transferring hyperspectral foundation models across sensors with differing spectral band configurations by proposing MBTI, a multi-branch parameter-efficient fine-tuning framework. The method partitions the hyperspectral image into contiguous spectral subsets, preserving full-spectrum information while employing a band-reuse mechanism to eliminate ineffective padding. Each branch is equipped with an independent LoRA module to enable efficient adaptation, and a channel attention mechanism is further introduced to adaptively fuse features from all branches. With only 2.33%–2.36% trainable parameters (using rank-8 LoRA), MBTI achieves significant performance gains over existing approaches on three public benchmarks, effectively balancing spectral continuity and cross-sensor transferability.

Abstract

Hyperspectral foundation models learn transferable spectral-spatial representations from large-scale unlabeled data. They provide an effective paradigm for adapting to downstream hyperspectral image (HSI) classification tasks with limited labeled samples. However, spectral band configurations vary substantially across sensors, which makes direct model transfer difficult. Existing adaptation strategies often compress, select, or reshape the original spectra to match model-specific input requirements. These operations may discard useful spectral information and weaken local spectral continuity. To address this problem, we propose MBTI, a Multi-Branch efficient fine-tuning framework for Hyperspectral Image classification. MBTI adapts hyperspectral foundation models to downstream classification tasks while preserving full-band spectral information. First, we introduce a spectral-continuity-preserving multi-branch preprocessing strategy. The original HSI is divided into multiple continuous spectral subsets, and a band reuse mechanism is used when the remaining bands cannot form a complete branch. This avoids invalid padding and unnecessary spectral loss. Second, independent Low-Rank Adaptation (LoRA) modules are inserted into each branch. They enable different spectral intervals to learn task-specific discriminative features while keeping most pre-trained parameters frozen. Finally, a multi-branch channel attention fusion module adaptively recalibrates and integrates features from all spectral branches. Experiments on three public hyperspectral datasets show that MBTI achieves competitive and superior performance compared with representative classification methods. Under the final rank-8 configuration, only about 2.33\%--2.36\% of the parameters are trainable. The code will be available at https://github.com/Azhenmiddleblock/MBTI/tree/main.

Problem

• hyperspectral image classification
• foundation models
• spectral band configuration
• spectral continuity
• model transfer

Innovation

• Multi-Branch Framework
• Spectral Continuity Preservation
• Low-Rank Adaptation (LoRA)
• Hyperspectral Image Classification
• Foundation Model Adaptation