HRSeg: High-Resolution Visual Perception and Enhancement for Reasoning Segmentation

2025-07-17

AI Summary

Existing vision-language segmentation methods suffer from coarse-grained perception and insufficient understanding of implicit instructions due to low-resolution pretraining of visual encoders; direct positional embedding interpolation yields marginal gains while incurring substantial computational overhead. To address these limitations, we propose a dual-module framework comprising High-Resolution Perception (HRP) and High-Resolution Enhancement (HRE). HRP enables local-global collaborative modeling via image patching and multi-granularity feature fusion. HRE improves text-mask semantic alignment through mask feature enhancement and high-resolution positional encoding. Our approach achieves state-of-the-art performance across multiple benchmarks while maintaining efficiency. Ablation studies confirm the efficacy of each module, demonstrating significant improvements in segmentation accuracy (+3.2 mIoU on average) and inference efficiency (−28% FLOPs).

Technology Category

• Computer Vision: Segmentation
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Multimodal Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The reasoning segmentation task involves segmenting objects within an image by interpreting implicit user instructions, which may encompass subtleties such as contextual cues and open-world knowledge. Despite significant advancements made by existing approaches, they remain constrained by low perceptual resolution, as visual encoders are typically pre-trained at lower resolutions. Furthermore, simply interpolating the positional embeddings of visual encoders to enhance perceptual resolution yields only marginal performance improvements while incurring substantial computational costs. To address this, we propose HRSeg, an efficient model with high-resolution fine-grained perception. It features two key innovations: High-Resolution Perception (HRP) and High-Resolution Enhancement (HRE). The HRP module processes high-resolution images through cropping, integrating local and global features for multi-granularity quality. The HRE module enhances mask features by integrating fine-grained information from high-resolution images, refining their alignment with text features for precise segmentation. Extensive ablation studies validate the effectiveness of our modules, while comprehensive experiments on multiple benchmark datasets demonstrate HRSeg's superior performance.

Problem

• Improving low perceptual resolution in reasoning segmentation
• Reducing computational costs of high-resolution image processing
• Enhancing alignment between image and text features

Innovation

• High-Resolution Perception for multi-granular quality
• High-Resolution Enhancement for precise segmentation
• Efficient model with fine-grained perception