Capacitive Touchscreens at Risk: Recovering Handwritten Trajectory on Smartphone via Electromagnetic Emanations

2025-12-12

AI Summary

This work uncovers a previously unknown non-contact electromagnetic (EM) side-channel vulnerability in capacitive touchscreens: their EM radiation inadvertently leaks fine-grained handwriting motion information. To exploit this, we propose TESLA—the first end-to-end real-time handwriting trajectory regression attack framework—integrating wideband EM signal acquisition, time-frequency feature extraction, lightweight temporal modeling (LSTM/TCN), and joint optimization of trajectory post-processing and character recognition. Evaluated on multiple mainstream commercial smartphones, TESLA achieves 77% character recognition accuracy and a 0.74 Jaccard similarity score, enabling high-fidelity reconstruction of continuous handwritten trajectories. This study is the first to empirically demonstrate that touchscreen EM emanations possess sufficient spatiotemporal resolution for precise trajectory recovery, thereby establishing a novel class of non-invasive side-channel threats. It provides critical empirical evidence for electromagnetic security assessment and mitigation in mobile devices.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Privacy
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

This paper reveals and exploits a critical security vulnerability: the electromagnetic (EM) side channel of capacitive touchscreens leaks sufficient information to recover fine-grained, continuous handwriting trajectories. We present Touchscreen Electromagnetic Side-channel Leakage Attack (TESLA), a non-contact attack framework that captures EM signals generated during on-screen writing and regresses them into two-dimensional (2D) handwriting trajectories in real time. Extensive evaluations across a variety of commercial off-the-shelf (COTS) smartphones show that TESLA achieves 77% character recognition accuracy and a Jaccard index of 0.74, demonstrating its capability to recover highly recognizable motion trajectories that closely resemble the original handwriting under realistic attack conditions.

Problem

• Recovering handwriting from smartphone touchscreen electromagnetic leaks.
• Exploiting EM side channels to reconstruct continuous writing trajectories.
• Assessing security risks of capacitive screens via non-contact attacks.

Innovation

• Exploits electromagnetic side-channel leakage from touchscreens
• Recovers handwriting trajectories via non-contact EM signal capture
• Achieves high recognition accuracy on commercial smartphones