Directing Mamba to Complex Textures: An Efficient Texture-Aware State Space Model for Image Restoration

2025-01-27

AI Summary

Existing high-resolution image restoration methods struggle to simultaneously model long-range dependencies, reconstruct fine-grained textures, and maintain computational efficiency. To address this, we propose TAMamba—a texture-aware state-space modeling framework—introducing, for the first time, a texture-adaptive modulated state transition matrix to dynamically capture degradation-space heterogeneity. We further design a low-overhead multi-directional perception block (MDPB) to enhance local-global collaborative representation in complex texture regions, and adapt the Mamba architecture for image restoration via lightweight structural redesign and degradation-sensitive feature gating. Evaluated on super-resolution, deraining, and low-light enhancement, TAMamba achieves state-of-the-art performance: it reduces parameter count by 37%, accelerates inference by 2.1×, and consistently surpasses existing efficient models in PSNR and SSIM.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Image restoration aims to recover details and enhance contrast in degraded images. With the growing demand for high-quality imaging ( extit{e.g.}, 4K and 8K), achieving a balance between restoration quality and computational efficiency has become increasingly critical. Existing methods, primarily based on CNNs, Transformers, or their hybrid approaches, apply uniform deep representation extraction across the image. However, these methods often struggle to effectively model long-range dependencies and largely overlook the spatial characteristics of image degradation (regions with richer textures tend to suffer more severe damage), making it hard to achieve the best trade-off between restoration quality and efficiency. To address these issues, we propose a novel texture-aware image restoration method, TAMambaIR, which simultaneously perceives image textures and achieves a trade-off between performance and efficiency. Specifically, we introduce a novel Texture-Aware State Space Model, which enhances texture awareness and improves efficiency by modulating the transition matrix of the state-space equation and focusing on regions with complex textures. Additionally, we design a {Multi-Directional Perception Block} to improve multi-directional receptive fields while maintaining low computational overhead. Extensive experiments on benchmarks for image super-resolution, deraining, and low-light image enhancement demonstrate that TAMambaIR achieves state-of-the-art performance with significantly improved efficiency, establishing it as a robust and efficient framework for image restoration.

Problem

• Image Restoration
• High Resolution
• Long-Distance Information

Innovation

• TAMambaIR
• Texture-aware Processing
• Efficient Multidirectional Information Extraction