Thalamus: A User Simulation Toolkit for Prototyping Multimodal Sensing Studies

2025-05-12

AI Summary

In human-computer interaction (HCI) research, multimodal physiological (e.g., EEG, ECG, GSR) and behavioral experiments face significant challenges—including high cost, cumbersome hardware debugging, and complex cross-device synchronization. To address these, this paper introduces the first cross-platform software framework enabling real-time simulation, dynamic signal modification, and nanosecond-precision temporal alignment of heterogeneous, geographically distributed physiological and behavioral devices. The framework employs signal-flow abstraction, device-agnostic APIs, a high-accuracy timestamp synchronization engine, and an integrated visualization interface—supporting hardware-free experimental rehearsal, protocol validation, and robustness testing. Its novel remote broadcast-based simulation mechanism drastically reduces experimental setup time. Deployed across multiple HCI laboratories, it has successfully accelerated pilot studies for eye-tracking–physiology hybrid paradigms. This work establishes a reusable, extensible, and standardized simulation infrastructure for multimodal user research.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Interaction Techniques and Devices
• Application Domains: Humanities & Computational Social Science

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

Conducting user studies that involve physiological and behavioral measurements is very time-consuming and expensive, as it not only involves a careful experiment design, device calibration, etc. but also a careful software testing. We propose Thalamus, a software toolkit for collecting and simulating multimodal signals that can help the experimenters to prepare in advance for unexpected situations before reaching out to the actual study participants and even before having to install or purchase a specific device. Among other features, Thalamus allows the experimenter to modify, synchronize, and broadcast physiological signals (as coming from various data streams) from different devices simultaneously and not necessarily located in the same place. Thalamus is cross-platform, cross-device, and simple to use, making it thus a valuable asset for HCI research.

Problem

• Simplifying multimodal sensing study preparation with simulation
• Reducing costs and time in user study experiments
• Enabling cross-platform synchronization of physiological signals

Innovation

• Simulates multimodal signals for user studies
• Modifies and synchronizes physiological data streams
• Cross-platform toolkit for HCI research