Snapshot Compressed Imaging Based Single-Measurement Computer Vision for Videos

2025-01-25

AI Summary

Snapshot compressive imaging (SCI) systems fail under ultra-low-light conditions (average photon count ≤ 20), where single-shot compressed measurements contain severe photon noise, hindering direct extraction of high-level visual cues—such as edges and depth—without explicit reconstruction. Method: We propose CompDAE, a compression-domain denoising autoencoder that explicitly models photon noise in the compressed domain. CompDAE integrates an end-to-end differentiable SCI forward model, an STFormer-based compressed-domain autoencoding architecture, and a multi-task joint loss, enabling unified optimization of image reconstruction, edge detection, and depth estimation. Contribution/Results: CompDAE bypasses explicit image reconstruction and directly performs high-level visual inference from a single compressed measurement. It significantly outperforms RGB-domain methods across multiple benchmarks and maintains robust performance under extremely low signal-to-noise ratios, overcoming a fundamental limitation of conventional SCI systems in weak-light scenarios.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Machine Learning: Learning on the Edge & Model Compression
• Data Mining & Knowledge Management: Data Compression

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Snapshot compressive imaging (SCI) is a promising technique for capturing high-speed video at low bandwidth and low power, typically by compressing multiple frames into a single measurement. However, similar to traditional CMOS image sensor based imaging systems, SCI also faces challenges in low-lighting photon-limited and low-signal-to-noise-ratio image conditions. In this paper, we propose a novel Compressive Denoising Autoencoder (CompDAE) using the STFormer architecture as the backbone, to explicitly model noise characteristics and provide computer vision functionalities such as edge detection and depth estimation directly from compressed sensing measurements, while accounting for realistic low-photon conditions. We evaluate the effectiveness of CompDAE across various datasets and demonstrated significant improvements in task performance compared to conventional RGB-based methods. In the case of ultra-low-lighting (APC $leq$ 20) while conventional methods failed, the proposed algorithm can still maintain competitive performance.

Problem

• Low-light Imaging
• Compressive Sensing
• Edge and Depth Extraction

Innovation

• Compressed Denoising Autoencoder (CompDAE)
• Low-light Imaging
• Image Understanding