SmartSplat: Feature-Smart Gaussians for Scalable Compression of Ultra-High-Resolution Images

2025-12-23

AI Summary

To address the challenge of efficient compression and real-time decoding of ultra-high-definition (UHD) images on resource-constrained edge devices, this paper proposes a feature-aware compression framework based on Gaussian Splatting. Our method introduces two novel sampling strategies—gradient-color joint variational sampling and exclusive uniform sampling—alongside scale-adaptive Gaussian shading initialization, enabling non-overlapping pixel-space coverage and cross-scale accurate color reconstruction. By jointly optimizing Gaussian positions, scales, and colors while incorporating image gradient and color variance features, the framework significantly enhances sparse representation capability for both local structures and global textures. Evaluated on DIV8K and a custom 16K dataset, our approach outperforms state-of-the-art methods under high compression ratios, achieving superior reconstruction fidelity. It further supports arbitrary-resolution adaptation, exhibits strong scalability, and demonstrates promising potential for on-device deployment.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Representation Learning for Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Recent advances in generative AI have accelerated the production of ultra-high-resolution visual content, posing significant challenges for efficient compression and real-time decoding on end-user devices. Inspired by 3D Gaussian Splatting, recent 2D Gaussian image models improve representation efficiency, yet existing methods struggle to balance compression ratio and reconstruction fidelity in ultra-high-resolution scenarios. To address this issue, we propose SmartSplat, a highly adaptive and feature-aware GS-based image compression framework that supports arbitrary image resolutions and compression ratios. SmartSplat leverages image-aware features such as gradients and color variances, introducing a Gradient-Color Guided Variational Sampling strategy together with an Exclusion-based Uniform Sampling scheme to improve the non-overlapping coverage of Gaussian primitives in pixel space. In addition, we propose a Scale-Adaptive Gaussian Color Sampling method to enhance color initialization across scales. Through joint optimization of spatial layout, scale, and color initialization, SmartSplat efficiently captures both local structures and global textures using a limited number of Gaussians, achieving high reconstruction quality under strong compression. Extensive experiments on DIV8K and a newly constructed 16K dataset demonstrate that SmartSplat consistently outperforms state-of-the-art methods at comparable compression ratios and exceeds their compression limits, showing strong scalability and practical applicability. The code is publicly available at https://github.com/lif314/SmartSplat.

Problem

• Compresses ultra-high-resolution images efficiently for end-user devices
• Balances compression ratio with reconstruction fidelity in high-resolution scenarios
• Improves Gaussian primitive coverage and color representation across scales

Innovation

• Uses gradient-color guided variational sampling for coverage
• Implements scale-adaptive Gaussian color sampling method
• Jointly optimizes spatial layout, scale, and color initialization