ContourFormer:Real-Time Contour-Based End-to-End Instance Segmentation Transformer

2025-01-29

AI Summary

To address the longstanding trade-off between accuracy and speed in real-time instance segmentation, this paper proposes the first end-to-end Transformer framework based on contour modeling. Methodologically, it abandons conventional mask-based representations and introduces a novel sub-contour decoupling mechanism coupled with boundary-aware adaptive sampling. Sub-contour modeling is driven by deformable attention, complemented by multi-stage progressive contour refinement and a serialized point representation of contours—all implemented within the DETR paradigm for efficient parallel decoding. A fine-grained contour distribution refinement technique further enhances boundary localization accuracy. Evaluated on SBD, COCO, and KINS benchmarks, the method achieves state-of-the-art performance at over 30 FPS—outperforming mainstream approaches including Mask R-CNN and QueryInst in both accuracy and inference speed.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper presents Contourformer, a real-time contour-based instance segmentation algorithm. The method is fully based on the DETR paradigm and achieves end-to-end inference through iterative and progressive mechanisms to optimize contours. To improve efficiency and accuracy, we develop two novel techniques: sub-contour decoupling mechanisms and contour fine-grained distribution refinement.In the sub-contour decoupling mechanism, we propose a deformable attention-based module that adaptively selects sampling regions based on the current predicted contour, enabling more effective capturing of object boundary information. Additionally, we design a multi-stage optimization process to enhance segmentation precision by progressively refining sub-contours. The contour fine-grained distribution refinement technique aims to further improve the ability to express fine details of contours.These innovations enable Contourformer to achieve stable and precise segmentation for each instance while maintaining real-time performance. Extensive experiments demonstrate the superior performance of Contourformer on multiple benchmark datasets, including SBD, COCO, and KINS. We conduct comprehensive evaluations and comparisons with existing state-of-the-art methods, showing significant improvements in both accuracy and inference speed.This work provides a new solution for contour-based instance segmentation tasks and lays a foundation for future research, with the potential to become a strong baseline method in this field.

Problem

• Real-time Image Recognition
• Efficient Segmentation
• Object Contour Detection

Innovation

• Contour Recognition
• Efficient Object Segmentation
• DETR Framework Enhancement