Semantic Interactivity: leveraging NLP to enable a shared interaction approach for joint activities

2025-11-07

AI Summary

Existing collaborative systems predominantly focus on individual tasks, limiting their capacity to support authentic joint activities and shared experiences. To address this, we propose “semantic interaction”—a novel paradigm that integrates natural language processing (NLP) techniques to enable group co-discussion within co-located spaces. We instantiate this paradigm through CollEagle, an interactive tabletop system that supports low-effort information creation, collective externalization, dynamic reorganization, and structured management of shared artifacts. Crucially, CollEagle embeds deep semantic understanding directly into the collaborative interface—marking a departure from traditional task-centric design approaches. This integration significantly enhances interaction quality and shared cognition among participants. Preliminary empirical evaluation demonstrates that semantic interaction effectively modulates collaborative behaviors, offering both theoretical grounding and practical design guidelines for collaboration interfaces targeting joint activity.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Humans and AI: Interaction Techniques and Devices
• Game Theory and Economic Paradigms: Coordination and Collaboration

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: The sharing economy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Collocated collaboration, where individuals work together in the same physical space and time, remains a cornerstone of effective teamwork. However, most collaborative systems are designed to support individual tasks rather than joint activities; they enable interactions for users to complete tasks rather than interactivity to engage in shared experiences. In this work, we introduce an NLP-driven mechanism that enables semantic interactivity through a shared interaction mechanism. This mechanism was developed as part of CollEagle, an interactive tabletop system that supports shared externalisation practices by offering a low-effort way for users to create, curate, organise, and structure information to capture the essence of collaborative discussions. Our preliminary study highlights the potential for semantic interactivity to mediate group interactions, suggesting that the interaction approach paves the way for designing novel collaborative interfaces. We contribute our implementation and offer insights for future research to enable semantic interactivity in systems that support joint activities.

Problem

• Developing NLP-driven semantic interactivity for shared collaborative experiences
• Creating low-effort interaction mechanisms for joint information structuring
• Enabling shared externalization practices in collocated teamwork systems

Innovation

• NLP-driven mechanism enables semantic interactivity
• Shared interaction approach for joint activities
• Interactive tabletop system supports collaborative externalization practices