DECKER: Domain-invariant Embedding for Cross-Keyboard Extraction and Recognition

2026-05-05

AI Summary

This work addresses the limited generalization of acoustic side-channel attacks across devices, users, and noisy environments by introducing HEAR, a large-scale, multidimensional dataset, and proposing the DECKER framework. DECKER learns domain-invariant keystroke embeddings through keyboard-specific feature normalization, domain-adversarial disentanglement, cross-keyboard contrastive alignment, and acoustic style randomization, followed by semantic post-processing using a large language model to refine recognition outputs. As the first systematic evaluation of acoustic keystroke inference under realistic generalization settings, this study demonstrates that DECKER substantially outperforms existing methods on the HEAR dataset, achieving marked performance gains especially in cross-keyboard and cross-user scenarios, thereby revealing the tangible threat posed by such attacks in real-world conditions.

Abstract

Acoustic side-channel attacks (ASCA) on keyboards pose a significant security risk, as keystrokes can be inferred from typing acoustics, revealing sensitive information. Prior ASCA studies are limited by small-scale datasets with restricted diversity in users, keyboards, and environments, constraining analysis across devices, microphones, and noise conditions. We introduce HEAR, a dataset designed to study ASCA along three axes: keyboard generalization, noise adaptation, and user bias. HEAR contains recordings from 53 participants using 37 laptop keyboards, collected in three realistic settings: (1) external microphone capture, (2) device microphone capture without network noise, and (3) VoIP-based streaming capture. This enables controlled evaluation across users, keyboards, and environments. On HEAR, we establish an ASCA benchmark spanning conventional features and pre-trained representations from raw audio and spectrograms in unimodal and multimodal settings. We propose DECKER, a domain-invariant keystroke inference framework with four stages: (1) Keyboard Signature Normalization to reduce device coloration, (2) domain-adversarial disentanglement to suppress keyboard identity, (3) supervised cross-keyboard contrastive alignment to enforce key consistency, and (4) Acoustic Style Randomization to synthesize unseen keyboard responses. We further explore sentence-level inference using an LLM-based post-processing layer to refine keystroke sequences via linguistic context. Results on HEAR show DECKER improves keystroke identification over strong baselines, particularly in cross-keyboard and cross-user settings, with further gains from language-model rectification. These findings highlight that ASCA remains effective across diverse users, devices, and noisy environments, underscoring its practical security risk.

Problem

• acoustic side-channel attack
• keyboard recognition
• cross-device generalization
• domain invariance
• keystroke inference

Innovation

• domain-invariant representation
• acoustic side-channel attack
• cross-keyboard generalization
• contrastive alignment
• acoustic style randomization