Pan-LUT: Efficient Pan-sharpening via Learnable Look-Up Tables

2025-03-31

AI Summary

To address the high computational cost and poor real-time deployability of deep learning methods in panchromatic sharpening of high-resolution remote sensing imagery, this paper proposes Pan-LUT, a lightweight learnable lookup table (LUT) framework. Its core innovations include the first introduction of three specialized LUT modules: PAN-guided LUT (PGLUT) for spectral mapping, Spatial Detail LUT (SDLUT) for local contextual modeling, and Adaptive Aggregation LUT (AALUT) for channel-wise adaptive fusion—collectively enabling efficient spectral fidelity preservation and spatial detail enhancement. With fewer than 300K parameters, Pan-LUT processes 8K-resolution images in under 1 ms on an RTX 2080 Ti GPU. Evaluated on full-resolution real-world scenes, it achieves state-of-the-art PSNR and SSIM scores, striking an unprecedented balance among spectral consistency, reconstruction accuracy, and edge-deployment efficiency.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recently, deep learning-based pan-sharpening algorithms have achieved notable advancements over traditional methods. However, many deep learning-based approaches incur substantial computational overhead during inference, especially with high-resolution images. This excessive computational demand limits the applicability of these methods in real-world scenarios, particularly in the absence of dedicated computing devices such as GPUs and TPUs. To address these challenges, we propose Pan-LUT, a novel learnable look-up table (LUT) framework for pan-sharpening that strikes a balance between performance and computational efficiency for high-resolution remote sensing images. To finely control the spectral transformation, we devise the PAN-guided look-up table (PGLUT) for channel-wise spectral mapping. To effectively capture fine-grained spatial details and adaptively learn local contexts, we introduce the spatial details look-up table (SDLUT) and adaptive aggregation look-up table (AALUT). Our proposed method contains fewer than 300K parameters and processes a 8K resolution image in under 1 ms using a single NVIDIA GeForce RTX 2080 Ti GPU, demonstrating significantly faster performance compared to other methods. Experiments reveal that Pan-LUT efficiently processes large remote sensing images in a lightweight manner, bridging the gap to real-world applications. Furthermore, our model surpasses SOTA methods in full-resolution scenes under real-world conditions, highlighting its effectiveness and efficiency.

Problem

• Reducing computational overhead in pan-sharpening algorithms
• Balancing performance and efficiency for high-resolution images
• Enabling real-world applicability without dedicated computing devices

Innovation

• Learnable LUT framework for pan-sharpening
• PAN-guided LUT for spectral mapping
• Spatial details LUT for fine-grained capture