Gaussian Splatting-based Low-Rank Tensor Representation for Multi-Dimensional Image Recovery

2025-11-18

AI Summary

Traditional tensor representations, such as t-SVD, suffer from coarse low-rank approximations and fixed orthogonal bases (e.g., DFT/DCT), limiting their ability to capture local high-frequency structures across spatial and channel dimensions in multidimensional images. To address this, we propose a Gaussian lattice-based low-rank tensor representation framework: (i) 2D Gaussian lattice embedding constructs an implicit low-rank tensor structure, enabling continuous and fine-grained modeling of spatial high-frequency details; (ii) 1D Gaussian lattice parameterization learns adaptive mode-3 transform bases, replacing rigid orthogonal bases. This work is the first to integrate Gaussian latticeization into both the structural design of tensor decomposition and the learning of transformation bases. Evaluated on unsupervised multidimensional image restoration tasks—including hyperspectral and multispectral image recovery—our method achieves superior high-frequency detail reconstruction accuracy, consistently outperforming state-of-the-art approaches.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Computer Vision: Representation Learning for Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Personalized content ranking and presentation

Abstract

Tensor singular value decomposition (t-SVD) is a promising tool for multi-dimensional image representation, which decomposes a multi-dimensional image into a latent tensor and an accompanying transform matrix. However, two critical limitations of t-SVD methods persist: (1) the approximation of the latent tensor (e.g., tensor factorizations) is coarse and fails to accurately capture spatial local high-frequency information; (2) The transform matrix is composed of fixed basis atoms (e.g., complex exponential atoms in DFT and cosine atoms in DCT) and cannot precisely capture local high-frequency information along the mode-3 fibers. To address these two limitations, we propose a Gaussian Splatting-based Low-rank tensor Representation (GSLR) framework, which compactly and continuously represents multi-dimensional images. Specifically, we leverage tailored 2D Gaussian splatting and 1D Gaussian splatting to generate the latent tensor and transform matrix, respectively. The 2D and 1D Gaussian splatting are indispensable and complementary under this representation framework, which enjoys a powerful representation capability, especially for local high-frequency information. To evaluate the representation ability of the proposed GSLR, we develop an unsupervised GSLR-based multi-dimensional image recovery model. Extensive experiments on multi-dimensional image recovery demonstrate that GSLR consistently outperforms state-of-the-art methods, particularly in capturing local high-frequency information.

Problem

• Improving coarse tensor approximations in multi-dimensional image representation
• Replacing fixed transform matrices to capture local high-frequency information
• Developing continuous representation for multi-dimensional image recovery

Innovation

• Gaussian splatting generates latent tensor
• Gaussian splatting creates transform matrix
• Continuous representation captures high-frequency details