Learning to Segment using Summary Statistics and Weak Supervision

2026-05-04

AI Summary

This work addresses the high cost of pixel-level annotation in medical image segmentation by proposing a novel approach that integrates diagnostic summary statistics—such as lesion area—with extremely sparse pixel-level weak supervision. The method employs a multi-objective loss function that jointly optimizes image reconstruction quality, fidelity to the provided summary statistics, and foreground overlap constraints derived from the sparse annotations. This design effectively overcomes the inherent limitations of relying solely on aggregate statistics, which often fail to yield precise spatial delineation. Evaluated on diverse datasets including natural images, breast ultrasound, and renal CT tumor scans, the proposed framework significantly outperforms baselines using only summary statistics. To the best of our knowledge, this is the first method to successfully synergize global statistical cues with minimal pixel-wise supervision, achieving high segmentation accuracy while substantially reducing annotation burden.

Abstract

Medical experts often manually segment images to obtain diagnostic statistics and discard the resulting annotations. We aim to train segmentation models to alleviate this burden, but constrained to the retained summary statistics (e.g., the area of the annotated region). Empirical results suggest that statistics alone are insufficient for this task, but adding weak information in the form of a few pixels within the area of interest significantly improves performance. We use a novel loss function that combines terms for image reconstruction quality, matching to summary statistics, and overlap between the predicted foreground and the weak supervisory signal. Experiments on standard image, ultrasound (breast cancer), and Computed Tomography (CT) scan (kidney tumors) data demonstrate the utility and potential of the approach.

Problem

• weak supervision
• image segmentation
• summary statistics
• medical imaging
• segmentation learning

Innovation

• weak supervision
• summary statistics
• image segmentation
• reconstruction loss
• medical imaging