Single Point, Full Mask: Velocity-Guided Level Set Evolution for End-to-End Amodal Segmentation

2025-08-03

AI Summary

Existing modality segmentation methods rely on costly strong prompts (e.g., masks or bounding boxes), lack geometric interpretability, and struggle with complex occlusions. To address these limitations, we propose the first speed-driven level-set evolution framework that performs end-to-end inference of fully occluded object shapes from a single point prompt. Our method employs a differentiable neural network to predict a shape-adaptive motion field, which guides the evolution of an implicit level-set function; initialization leverages both image features and the point prompt, enabling end-to-end training. The framework inherently supports topological flexibility while providing explicit geometric interpretability—overcoming the black-box nature of mask regression. Evaluated on COCOA-cls, D2SA, and KINS, our approach with only a single-point input surpasses state-of-the-art strong-prompt methods, significantly improving occlusion recovery accuracy and validating the efficacy of weakly supervised geometric modeling.

Technology Category

• Computer Vision: Segmentation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Multimodal Learning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Amodal segmentation aims to recover complete object shapes, including occluded regions with no visual appearance, whereas conventional segmentation focuses solely on visible areas. Existing methods typically rely on strong prompts, such as visible masks or bounding boxes, which are costly or impractical to obtain in real-world settings. While recent approaches such as the Segment Anything Model (SAM) support point-based prompts for guidance, they often perform direct mask regression without explicitly modeling shape evolution, limiting generalization in complex occlusion scenarios. Moreover, most existing methods suffer from a black-box nature, lacking geometric interpretability and offering limited insight into how occluded shapes are inferred. To deal with these limitations, we propose VELA, an end-to-end VElocity-driven Level-set Amodal segmentation method that performs explicit contour evolution from point-based prompts. VELA first constructs an initial level set function from image features and the point input, which then progressively evolves into the final amodal mask under the guidance of a shape-specific motion field predicted by a fully differentiable network. This network learns to generate evolution dynamics at each step, enabling geometrically grounded and topologically flexible contour modeling. Extensive experiments on COCOA-cls, D2SA, and KINS benchmarks demonstrate that VELA outperforms existing strongly prompted methods while requiring only a single-point prompt, validating the effectiveness of interpretable geometric modeling under weak guidance. The code will be publicly released.

Problem

• Amodal segmentation recovers complete object shapes including occluded regions
• Existing methods need costly strong prompts like visible masks or boxes
• Current approaches lack geometric interpretability in occluded shape inference

Innovation

• Velocity-driven level-set evolution for amodal segmentation
• Single-point prompt for weak guidance
• Differentiable network predicts shape-specific motion field