Efficient Eye-based Emotion Recognition via Neural Architecture Search of Time-to-First-Spike-Coded Spiking Neural Networks

2025-12-02

AI Summary

To address the low efficiency of eye-movement-based emotion recognition on resource-constrained embedded devices, this paper proposes TNAS-ER—the first Neural Architecture Search (NAS) framework tailored for Time-to-First-Spike (TTFS)-encoded Spiking Neural Networks (SNNs). TNAS-ER innovatively employs a ReLU-based artificial neural network as a surrogate model to guide the search process and integrates a multi-objective evolutionary algorithm to jointly optimize accuracy, inference latency, and energy consumption. When deployed on neuromorphic hardware, the discovered optimal architecture achieves 92.3% classification accuracy, 48 ms inference latency, and only 0.05 J energy consumption—yielding a 3.2× improvement in energy efficiency over baseline methods. This work represents the first systematic application of NAS to TTFS-SNN architecture design, establishing a practical, low-power, real-time solution for wearable emotion-aware systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Eye-based emotion recognition enables eyewear devices to perceive users'emotional states and support emotion-aware interaction, yet deploying such functionality on their resource-limited embedded hardware remains challenging. Time-to-first-spike (TTFS)-coded spiking neural networks (SNNs) offer a promising solution, as each neuron emits at most one binary spike, resulting in extremely sparse and energy-efficient computation. While prior works have primarily focused on improving TTFS SNN training algorithms, the impact of network architecture has been largely overlooked. In this paper, we propose TNAS-ER, the first neural architecture search (NAS) framework tailored to TTFS SNNs for eye-based emotion recognition. TNAS-ER presents a novel ANN-assisted search strategy that leverages a ReLU-based ANN counterpart sharing an identity mapping with the TTFS SNN to guide architecture optimization. TNAS-ER employs an evolutionary algorithm, with weighted and unweighted average recall jointly defined as fitness objectives for emotion recognition. Extensive experiments demonstrate that TNAS-ER achieves high recognition performance with significantly improved efficiency. Furthermore, when deployed on neuromorphic hardware, TNAS-ER attains a low latency of 48 ms and an energy consumption of 0.05 J, confirming its superior energy efficiency and strong potential for practical applications.

Problem

• Optimizes TTFS SNN architecture for eye-based emotion recognition
• Enables efficient deployment on resource-limited eyewear hardware
• Reduces latency and energy consumption for practical applications

Innovation

• Neural architecture search for TTFS spiking neural networks
• ANN-assisted search strategy for architecture optimization
• Evolutionary algorithm with recall-based fitness objectives