AnalogNAS-Bench: A NAS Benchmark for Analog In-Memory Computing

2025-06-23

AI Summary

Current neural networks are not optimized for analog in-memory computing (AIMC) hardware non-idealities—such as device noise and time-varying drift—leading to significant accuracy degradation; meanwhile, mainstream neural architecture search (NAS) methods lack AIMC-aware evaluation benchmarks. This work introduces the first dedicated NAS benchmark for AIMC, integrating realistic hardware non-ideality modeling with diverse network topology search to enable analog-aware automated architecture design. Experimental analysis uncovers three key insights: (1) standard quantization fails to capture AIMC-specific noise characteristics; (2) wider parallel branch structures exhibit superior robustness; and (3) skip connections markedly enhance resilience against time-varying drift. The benchmark is publicly released, providing a reproducible, scalable evaluation platform and architectural guidance for AIMC-customized model development.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Analog In-memory Computing (AIMC) has emerged as a highly efficient paradigm for accelerating Deep Neural Networks (DNNs), offering significant energy and latency benefits over conventional digital hardware. However, state-of-the-art neural networks are not inherently designed for AIMC, as they fail to account for its unique non-idealities. Neural Architecture Search (NAS) is thus needed to systematically discover neural architectures optimized explicitly for AIMC constraints. However, comparing NAS methodologies and extracting insights about robust architectures for AIMC requires a dedicated NAS benchmark that explicitly accounts for AIMC-specific hardware non-idealities. To address this, we introduce AnalogNAS-Bench, the first NAS benchmark tailored specifically for AIMC. Our study reveals three key insights: (1) standard quantization techniques fail to capture AIMC-specific noises, (2) robust architectures tend to feature wider and branched blocks, (3) skip connections improve resilience to temporal drift noise. These insights highlight the limitations of current NAS benchmarks for AIMC and pave the way for future analog-aware NAS. All the implementations used in this paper can be found at https://github.com/IBM/analog-nas/tree/main/analognasbench.

Problem

• Neural networks not designed for AIMC non-idealities
• Lack of dedicated NAS benchmark for AIMC constraints
• Standard quantization fails to capture AIMC-specific noises

Innovation

• Introduces AnalogNAS-Bench for AIMC-specific NAS benchmarking
• Reveals wider branched blocks enhance AIMC robustness
• Shows skip connections mitigate temporal drift noise