NeuResonance: Exploring Feedback Experiences for Fostering the Inter-brain Synchronization

2025-04-25

International Conference on Human Factors in Computing Systems

AI Summary

This study investigates whether multimodal real-time feedback—visual, auditory, and tactile—can enhance inter-brain synchronization (IBS) to foster prosocial affect (e.g., trust, empathy, and interpersonal closeness), thereby supporting clinical therapist–patient rehabilitation, teacher–student motor instruction, and collaborative group art practices that critically depend on strong interpersonal coupling. We systematically compare, for the first time, the modality-specific regulatory effects of these three sensory channels on IBS and propose design principles for multimodal feedback targeting IBS augmentation. A closed-loop neurofeedback system was implemented using concurrent EEG recording, surface-projected visual mapping, chord-based sonification, and wrist-mounted vibrotactile stimulation. Experimental results demonstrate significant increases in both IBS magnitude and subjective ratings of cooperative experience and emotional connection across all modalities, validating the efficacy and feasibility of multimodal neurofeedback for strengthening interpersonal coordination.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

When several individuals collaborate on a shared task, their brain activities often synchronize. This phenomenon, known as Inter-brain Synchronization (IBS), is notable for inducing prosocial outcomes such as enhanced interpersonal feelings, including closeness, trust, empathy, and more. Further strengthening the IBS with the aid of external feedback would be beneficial for scenarios where those prosocial feelings play a vital role in interpersonal communication, such as rehabilitation between a therapist and a patient, motor skill learning between a teacher and a student, and group performance art. This paper investigates whether visual, auditory, and haptic feedback of the IBS level can further enhance its intensity, offering design recommendations for feedback systems in IBS. We report findings when three different types of feedback were provided: IBS level feedback by means of on-body projection mapping, sonification using chords, and vibration bands attached to the wrist.

Problem

• Enhancing inter-brain synchronization through multi-sensory feedback systems
• Investigating visual auditory haptic feedback effects on brain synchrony
• Designing IBS feedback for therapeutic educational artistic applications

Innovation

• Visual feedback using on-body projection mapping
• Auditory feedback through chord sonification
• Haptic feedback with wrist vibration bands