FLIM-based Salient Object Detection Networks with Adaptive Decoders

2025-04-29

AI Summary

To address the dual challenges of model lightweighting and ultra-few-shot training for salient object detection (SOD) in resource-constrained scenarios, this paper proposes FLIM: a flyweight SOD network based on image-token feature learning. Methodologically, FLIM eliminates backpropagation by training exclusively on 3–4 unlabeled representative images; introduces a novel pixel-wise adaptive decoder that generates neuron-specific weights for each pixel; and employs token-guided feature kernel initialization coupled with a gradient-free parameter estimation mechanism. With only 1% of the parameters of existing lightweight SOD models, FLIM achieves an ultra-compact “flyweight” architecture. Evaluated on two challenging SOD benchmarks, FLIM consistently outperforms three state-of-the-art lightweight models and ablated variants, demonstrating the effectiveness and strong generalization capability of its unsupervised weight generation paradigm under ultra-few-shot and zero-label constraints.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Salient Object Detection (SOD) methods can locate objects that stand out in an image, assign higher values to their pixels in a saliency map, and binarize the map outputting a predicted segmentation mask. A recent tendency is to investigate pre-trained lightweight models rather than deep neural networks in SOD tasks, coping with applications under limited computational resources. In this context, we have investigated lightweight networks using a methodology named Feature Learning from Image Markers (FLIM), which assumes that the encoder's kernels can be estimated from marker pixels on discriminative regions of a few representative images. This work proposes flyweight networks, hundreds of times lighter than lightweight models, for SOD by combining a FLIM encoder with an adaptive decoder, whose weights are estimated for each input image by a given heuristic function. Such FLIM networks are trained from three to four representative images only and without backpropagation, making the models suitable for applications under labeled data constraints as well. We study five adaptive decoders; two of them are introduced here. Differently from the previous ones that rely on one neuron per pixel with shared weights, the heuristic functions of the new adaptive decoders estimate the weights of each neuron per pixel. We compare FLIM models with adaptive decoders for two challenging SOD tasks with three lightweight networks from the state-of-the-art, two FLIM networks with decoders trained by backpropagation, and one FLIM network whose labeled markers define the decoder's weights. The experiments demonstrate the advantages of the proposed networks over the baselines, revealing the importance of further investigating such methods in new applications.

Problem

• Develops flyweight networks for Salient Object Detection (SOD).
• Uses FLIM encoders and adaptive decoders without backpropagation.
• Addresses SOD under limited computational and labeled data constraints.

Innovation

• FLIM encoder with adaptive decoder
• Heuristic function estimates neuron weights
• Training without backpropagation from few images