Language-Based Swarm Perception: Decentralized Person Re-Identification via Natural Language Descriptions

2026-01-18

AI Summary

This work addresses the lack of interpretability and natural language interaction in decentralized person re-identification for robotic swarms by proposing a novel approach based on vision-language models (VLMs). The method translates individual visual appearances into natural language descriptions, enabling collaborative re-identification through textual comparison and clustering within a decentralized architecture. Furthermore, it leverages large language models to generate concise, high-level summaries of collective perception. By establishing natural language as the core representational modality for swarm awareness, the approach facilitates semantic-level communication and queryable human-swarm interaction, significantly enhancing system transparency and human-robot collaboration potential. Preliminary experiments demonstrate that the method achieves identity consistency and interpretability on par with conventional embedding-based approaches, although challenges remain in optimizing text similarity computation and computational overhead.

Technology Category

• Computer Vision: Language and Vision
• Intelligent Robots: Human-Robot Interaction
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We introduce a method for decentralized person re-identification in robot swarms that leverages natural language as the primary representational modality. Unlike traditional approaches that rely on opaque visual embeddings -- high-dimensional feature vectors extracted from images -- the proposed method uses human-readable language to represent observations. Each robot locally detects and describes individuals using a vision-language model (VLM), producing textual descriptions of appearance instead of feature vectors. These descriptions are compared and clustered across the swarm without centralized coordination, allowing robots to collaboratively group observations of the same individual. Each cluster is distilled into a representative description by a language model, providing an interpretable, concise summary of the swarm's collective perception. This approach enables natural-language querying, enhances transparency, and supports explainable swarm behavior. Preliminary experiments demonstrate competitive performance in identity consistency and interpretability compared to embedding-based methods, despite current limitations in text similarity and computational load. Ongoing work explores refined similarity metrics, semantic navigation, and the extension of language-based perception to environmental elements. This work prioritizes decentralized perception and communication, while active navigation remains an open direction for future study.

Problem

• decentralized person re-identification
• robot swarms
• natural language descriptions
• swarm perception
• collaborative perception

Innovation

• language-based perception
• decentralized person re-identification
• vision-language model
• interpretable swarm intelligence
• natural language description