MSDS: Deep Structural Similarity with Multiscale Representation

2026-04-21

AI Summary

Existing deep perceptual similarity models are predominantly limited to single-scale analysis, overlooking the critical role of multi-scale structural information in image quality assessment. This work addresses this limitation by explicitly modeling spatial scale as an independent variable and introduces a minimalist multi-scale framework: DeepSSIM is computed independently at each level of a feature pyramid, and the resulting scores are aggregated via lightweight, learnable global weights. Despite introducing minimal computational overhead, the proposed method significantly outperforms single-scale baselines across multiple benchmark datasets, thereby demonstrating both the effectiveness and necessity of explicit multi-scale modeling in deep perceptual similarity estimation.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Multiagent Systems: Other Foundations of Multi Agent Systems

Application Category

• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Models for Web evolution
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Deep-feature-based perceptual similarity models have demonstrated strong alignment with human visual perception in Image Quality Assessment (IQA). However, most existing approaches operate at a single spatial scale, implicitly assuming that structural similarity at a fixed resolution is sufficient. The role of spatial scale in deep-feature similarity modeling thus remains insufficiently understood. In this letter, we isolate spatial scale as an independent factor using a minimal multiscale extension of DeepSSIM, referred to as Deep Structural Similarity with Multiscale Representation (MSDS). The proposed framework decouples deep feature representation from cross-scale integration by computing DeepSSIM independently across pyramid levels and fusing the resulting scores with a lightweight set of learnable global weights. Experiments on multiple benchmark datasets demonstrate consistent and statistically significant improvements over the single-scale baseline, while introducing negligible additional complexity. The results empirically confirm spatial scale as a non-negligible factor in deep perceptual similarity, isolated here via a minimal testbed.

Problem

• perceptual similarity
• spatial scale
• image quality assessment
• deep features
• multiscale representation

Innovation

• multiscale representation
• deep structural similarity
• perceptual similarity
• image quality assessment
• spatial scale