GAPNet: A Lightweight Framework for Image and Video Salient Object Detection via Granularity-Aware Paradigm

2025-08-10

AI Summary

To address the high computational cost of salient object detection (SOD) models and their difficulty in deployment on edge devices, this paper proposes GAPNet, a lightweight SOD framework. Its core innovation is a granularity-aware paradigm: it introduces Granularity Pyramid Convolution (GPC) and Cross-Scale Attention (CSA) modules to jointly exploit coarse-grained semantic cues and fine-grained boundary supervision, enabling efficient multi-scale feature decoding and global contextual modeling; additionally, it designs a lightweight self-attention-based decoder. Evaluated on multiple standard benchmarks, GAPNet achieves state-of-the-art performance among lightweight SOD models—delivering superior accuracy with significantly fewer parameters and FLOPs. This balance of high precision and computational efficiency makes GAPNet particularly suitable for resource-constrained edge scenarios.

Technology Category

• Computer Vision: Object Detection & Categorization
• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT

Abstract

Recent salient object detection (SOD) models predominantly rely on heavyweight backbones, incurring substantial computational cost and hindering their practical application in various real-world settings, particularly on edge devices. This paper presents GAPNet, a lightweight network built on the granularity-aware paradigm for both image and video SOD. We assign saliency maps of different granularities to supervise the multi-scale decoder side-outputs: coarse object locations for high-level outputs and fine-grained object boundaries for low-level outputs. Specifically, our decoder is built with granularity-aware connections which fuse high-level features of low granularity and low-level features of high granularity, respectively. To support these connections, we design granular pyramid convolution (GPC) and cross-scale attention (CSA) modules for efficient fusion of low-scale and high-scale features, respectively. On top of the encoder, a self-attention module is built to learn global information, enabling accurate object localization with negligible computational cost. Unlike traditional U-Net-based approaches, our proposed method optimizes feature utilization and semantic interpretation while applying appropriate supervision at each processing stage. Extensive experiments show that the proposed method achieves a new state-of-the-art performance among lightweight image and video SOD models. Code is available at https://github.com/yuhuan-wu/GAPNet.

Problem

• Lightweight SOD model for edge devices
• Granularity-aware multi-scale saliency detection
• Efficient feature fusion with low computational cost

Innovation

• Lightweight granularity-aware network for SOD
• Granular pyramid convolution for feature fusion
• Self-attention module for global information