Hybrid Photonic-digital Accelerator for Attention Mechanism

2025-01-20

AI Summary

Transformer attention mechanisms are computationally intensive, leading to high energy consumption and latency bottlenecks—particularly in photonic accelerators, where electro-optic/opto-electronic conversion overhead severely limits efficiency. This work proposes a photonic-electronic hybrid hardware accelerator tailored for the attention core. It introduces, for the first time, an adaptive low-/high-resolution conversion mechanism based on dynamic signal classification comparators, and achieves co-optimization of photonic computing units and digital circuits in both area and latency. Key innovations include a photonic matrix multiplication unit, mixed-precision ADC/DACs, custom approximate digital circuits, and dynamic-threshold comparators. Compared to state-of-the-art photonic Transformer accelerators, the design achieves 42 TOPS/W energy efficiency at sequence length 128, with 9.8× improvement in performance-per-area and 2.2× improvement in energy-efficiency-per-area.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: Low Level & Physics-based Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT

Abstract

The wide adoption and substantial computational resource requirements of attention-based Transformers have spurred the demand for efficient hardware accelerators. Unlike digital-based accelerators, there is growing interest in exploring photonics due to its high energy efficiency and ultra-fast processing speeds. However, the significant signal conversion overhead limits the performance of photonic-based accelerators. In this work, we propose HyAtten, a photonic-based attention accelerator with minimize signal conversion overhead. HyAtten incorporates a signal comparator to classify signals into two categories based on whether they can be processed by low-resolution converters. HyAtten integrates low-resolution converters to process all low-resolution signals, thereby boosting the parallelism of photonic computing. For signals requiring high-resolution conversion, HyAtten uses digital circuits instead of signal converters to reduce area and latency overhead. Compared to state-of-the-art photonic-based Transformer accelerator, HyAtten achieves 9.8X performance/area and 2.2X energy-efficiency/area improvement.

Problem

• Transformer models
• computational resources
• hardware accelerators

Innovation

• HyAtten
• signal comparator
• energy efficiency