U-CFR: Uncertainty-Guided Cascade Forward Refinement for Interactive Segmentation

2026-07-22

AI Summary

This work addresses the limitations of existing interactive image segmentation methods, which typically require numerous user clicks and exhibit slow convergence. The authors propose a self-correction framework operating at inference time that automatically generates internal pseudo-clicks by integrating segmentation uncertainty, contour gradients, and explicit edge predictions into a boundary-aware uncertainty scoring mechanism. A dual-head network architecture with a shared encoder–decoder is introduced to jointly optimize region consistency and boundary sharpness, enabling a cascaded forward refinement process. This approach substantially improves initial mask quality, boundary accuracy, and click efficiency, reducing the number of required user interactions by over 10% on challenging benchmarks such as the Berkeley dataset.

Abstract

Interactive image segmentation is critical for efficient image annotation; however, existing methods often require many corrective clicks or rely on passive refinement schemes that converge slowly. We propose Uncertainty-Guided Cascade Forward Refinement (U-CFR), a novel inference-time framework that enables models to autonomously self-correct after each user interaction. U-CFR introduces a boundary-aware uncertainty score that fuses segmentation uncertainty, contour gradients, and explicit edge predictions to guide the placement of internal pseudo-clicks. These self-generated clicks target the most ambiguous boundary regions, providing strong corrective signals without additional manual input. To support this process, we design a dual-head network with a shared encoder-decoder backbone: a segmentation head ensures region consistency, while an edge head sharpens boundary alignment. In inference, U-CFR launches a cascade of refinement steps, where each stage leverages the uncertainty-driven pseudo-clicks to refine the mask progressively. Experiments on standard benchmark datasets demonstrate that the proposed U-CFR improves click efficiency, initial mask quality, and boundary accuracy. It reduces the required clicks by over 10% on challenging datasets like Berkeley and offers a more intelligent and efficient interactive annotation.

Problem

• interactive segmentation
• click efficiency
• boundary accuracy
• uncertainty estimation
• image annotation

Innovation

• interactive segmentation
• uncertainty-guided refinement
• pseudo-clicks
• boundary-aware uncertainty
• cascade refinement