Towards Spatially-Varying Gain and Binning

2025-07-05

AI Summary

Pixel miniaturization degrades photon collection efficiency, leading to increased noise and constrained dynamic range (DR). To address this, we propose a synergistic optimization framework integrating spatially adaptive analog gain control and pixel binning. First, we introduce spatially varying analog gain to suppress readout noise near zero and extend sensor DR by an order of magnitude. Second, we formulate an analytical model for optimal binning size selection based on local luminance, demonstrating that digital binning outperforms conventional analog binning in signal-to-noise ratio—especially under high-gain conditions. Finally, we unify high-dynamic-range (HDR) fusion, vignetting correction, and geometric distortion compensation into an end-to-end imaging enhancement pipeline. Experimental results show substantial improvements in both SNR and DR, with consistent visual quality enhancement across challenging illumination conditions.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Mixed Discrete/Continuous Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Privacy-enhancing technologies
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Pixels in image sensors have progressively become smaller, driven by the goal of producing higher-resolution imagery. However, ceteris paribus, a smaller pixel accumulates less light, making image quality worse. This interplay of resolution, noise, and the dynamic range of the sensor and their impact on the eventual quality of acquired imagery is a fundamental concept in photography. In this paper, we propose spatially-varying gain and binning to enhance the noise performance and dynamic range of image sensors. First, we show that by varying gain spatially to local scene brightness, the read noise can be made negligible, and the dynamic range of a sensor is expanded by an order of magnitude. Second, we propose a simple analysis to find a binning size that best balances resolution and noise for a given light level; this analysis predicts a spatially-varying binning strategy, again based on local scene brightness, to effectively increase the overall signal-to-noise ratio. % without sacrificing resolution. We discuss analog and digital binning modes and, perhaps surprisingly, show that digital binning outperforms its analog counterparts when a larger gain is allowed. Finally, we demonstrate that combining spatially-varying gain and binning in various applications, including high dynamic range imaging, vignetting, and lens distortion.

Problem

• Enhancing noise performance in small-pixel image sensors
• Expanding dynamic range via spatially-varying gain
• Optimizing binning size for resolution-noise balance

Innovation

• Spatially-varying gain enhances dynamic range
• Local brightness-based binning balances resolution
• Digital binning outperforms analog with gain