Problem
Research questions and friction points this paper is trying to address.
Guideline-Consistent Segmentation
Multi-Agent Refinement
Stateless Feedback
InsightSeg: Reusing Correction Insights for Guideline-Consistent Segmentation
🤖 AI Summary
本文提出InsightSeg，通过重用校正见解来解决指南一致的分割问题，利用视觉概念向量将成功的校正案例转化为可复用的见解，提高分割质量并减少修正步骤。
📝 Abstract
Guideline-consistent semantic segmentation requires more than category recognition, as real-world labeling policies demand fine-grained, task-specific decisions. Recent multi-agent refinement systems improve compliance with such textual guidelines by detecting and correcting errors. However, they are stateless: feedback from the critiquing agent is discarded, causing the same guideline-specific mistakes to be repeatedly rediscovered and corrected across the dataset at the cost of additional refinement. We introduce InsightSeg, an episodic memory mechanism that converts successful correction episodes into reusable, visually grounded insights. A meta-analyzer distills each qualifying episode into directive natural-language insights and anchors them to the local image regions that caused the error using patch-level visual concept vectors. On subsequent images, these concepts are matched against dense patch embeddings to retrieve relevant insights, which condition the segmenting agent before making its first prediction. This shifts the system from correcting recurring errors to preventing them, improving segmentation quality before any refinement occurs. Across Waymo and Cityscapes, InsightSeg improves both first-pass and final guideline-consistent segmentation performance while requiring fewer refinement steps, demonstrating that multi-agent refinement can become more accurate and efficient by drawing on past correction experience.
Innovation
Methods, ideas, or system contributions that make the work stand out.
episodic memory
visually grounded insights
meta-analyzer
patch-level visual concept vectors
guideline-consistent segmentation
🔎 Similar Papers
No similar papers found.
