A Study on Human-Swarm Interaction: A Framework for Assessing Situation Awareness and Task Performance

2025-03-19

AI Summary

This work addresses the challenge of jointly evaluating situational awareness (SA) and task performance in human–swarm interaction under dynamic conditions. We propose the first systematic assessment framework integrating all three levels of SA—perception, comprehension, and projection—through a tablet-based single-target search experiment. Subjective SA (measured via SAGAT), objective SA (assessed via real-time probe injections), and swarm behavioral metrics—including centroid activity location and distance to target—are concurrently collected and analyzed quantitatively. Innovatively, we identify “centroid-active robot position” as a valid proxy metric for SA. Results demonstrate that perception and projection capabilities predominantly drive task effectiveness; furthermore, the interface exhibits learnability, with significant performance improvement across training trials. These findings provide both theoretical foundations and empirical evidence for designing high-SA human–swarm collaborative interfaces.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Human-Robot Interaction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper introduces a framework for human swarm interaction studies that measures situation awareness in dynamic environments. A tablet-based interface was developed for a user study by implementing the concepts introduced in the framework, where operators guided a robotic swarm in a single-target search task, marking hazardous cells unknown to the swarm. Both subjective and objective situation awareness measures were used, with task performance evaluated based on how close the robots were to the target. The framework enabled a structured investigation of the role of situation awareness in human swarm interaction, leading to key findings such as improved task performance across attempts, showing the interface was learnable, centroid active robot position proved to be a useful task performance metric for assessing situation awareness, perception and projection played a key role in task performance, highlighting their importance in interface design and both subjective and objective situation awareness influenced task performance, emphasizing the need for interfaces that support both. These findings validate our framework as a structured approach for integrating situation awareness concepts into human swarm interaction studies, offering a systematic way to assess situation awareness and task performance. The framework can be applied to other swarming studies to evaluate interface learnability, identify meaningful task performance metrics, and refine interface designs to enhance situation awareness, ultimately improving human swarm interaction in dynamic environments.

Problem

• Measures situation awareness in human-swarm interaction.
• Evaluates task performance using subjective and objective metrics.
• Develops a framework to improve interface design for dynamic environments.

Innovation

• Tablet-based interface for human-swarm interaction studies
• Combines subjective and objective situation awareness measures
• Framework integrates situation awareness into swarm task performance