SLAP: Selective Local Vision-Language Alignment for Fish Re-Identification via Partial Optimal Transport

2026-08-09

AI Summary

This work addresses the limitations of existing CLIP-based methods in fish re-identification, which rely on global image-text alignment and are thus susceptible to background clutter and non-discriminative regions. To overcome this, the authors propose a selective local visual-language alignment framework that, for the first time, incorporates partial optimal transport (POT) into fine-grained fish re-identification. By leveraging POT, the method establishes selective correspondences between local image patch embeddings and multiple identity-aware textual prompts, thereby enhancing strongly relevant cross-modal associations while suppressing interference from irrelevant regions. Built upon the CLIP architecture and integrating local embedding, multi-prompt encoding, and end-to-end training, the proposed approach significantly outperforms existing CLIP-based methods on the Symphodus melops dataset and demonstrates superior generalization across multiple marine species re-identification benchmarks.

Abstract

Individual fish re-identification (ReID) is a fine-grained recognition problem in which identity-discriminative cues are often localized to specific body regions rather than distributed uniformly across the animal. Nevertheless, recent CLIP-based ReID methods rely predominantly on global image-text alignment, allowing background and weakly discriminative regions to contribute to cross-modal supervision. We propose a selective local vision-language alignment framework that establishes localized correspondences between visual patch embeddings and multiple identity-aware prompt embeddings through Partial Optimal Transport (POT). Rather than enforcing exhaustive correspondence, POT enables selective matching between visual patches and prompt embeddings, allowing the model to emphasize the strongest cross-modal correspondences while avoiding forced alignment of weakly matching regions, thereby yielding more discriminative visual representations for retrieval. The framework is trained end-to-end, while only the adapted visual encoder is retained during inference. Experiments on the longitudinal Symphodus melops dataset demonstrate consistent improvements over recent CLIP-based ReID methods under both closed-set and open-set evaluation protocols. Additional evaluations on other datasets further demonstrate the generalization capability of the proposed method across diverse marine ReID benchmarks.

Problem

• fish re-identification
• fine-grained recognition
• local discriminative cues
• vision-language alignment
• cross-modal supervision

Innovation

• Selective Local Alignment
• Partial Optimal Transport
• Vision-Language Alignment
• Fish Re-Identification
• CLIP-based ReID