🤖 AI Summary
Transformer decoding is hindered by the high energy consumption and latency associated with attention computation and KV cache data movement. This work proposes a ferroelectric charge-domain computing unit (FCDC), which, for the first time, leverages HZO-based ferroelectric capacitors to perform non-volatile charge-domain vector-matrix multiplication. The design supports both full-die integration and a dedicated KV coprocessor deployment mode, augmented by a refresh-free KV cache residency mechanism. Through device-to-system co-modeling and multi-scale simulation across twelve pretrained large language models, the approach demonstrates minimal degradation—perplexity increases by no more than 2.9% and attention accuracy loss remains below 1% for models of at least 7B parameters. Compared to GPU baselines, it achieves 1.36–35× higher energy efficiency in RAG and agent tasks, and exceeds 41× in conversational residency scenarios.
📝 Abstract
Transformer decoding is constrained by both attention compute and KV-cache movement. This paper presents the Ferroelectric Charge-Domain Compute Cell (FCDC), a hafnium-zirconium-oxide (HZO) memcapacitor with an access device that stores analog state nonvolatilely and performs charge-domain VMM for attention. Two deployment modes are evaluated throughout: a full-substrate mode that runs q, k, v, o projections and both attention matmuls on FCDC, and a KV-coprocessor mode that only stores KV and executes the two attention matmuls; the projection-noise budget upper-bounds the coprocessor mode.
The device-to-system model is cross-checked across ngspice, CrossSim, FiPy, and NeuroSim and anchored in recent wafer-scale 10 nm HZO measurements. Across 12 pretrained LLMs (1.1-32 B dense, plus a partial-layer Mixtral-8x22B 141 B-MoE stress test at k=75% and a 128 k-context dense-Mistral replication), all-layer noise substitution adds only +2.62% WikiText-2 perplexity on Qwen3-32B and +2.90% +/- 0.33% on Mistral-7B-v0.3 (five-seed mean). End-to-end analog attention adds at most +1.68 pp on TinyLlama-1.1B and shrinks below +/-1 pp on every >=7 B model. Downstream accuracy on HellaSwag, ARC, LAMBADA, and GSM8K stays within 5% of the digital baseline for Mistral-7B (MMLU -1.6 pp).
The headline energy win is nonvolatility, no refresh, and KV-cache residency. A workload-level simulator anchored on measured INT4 decode energy delivers 18-35x lower per-served-token energy on RAG and agent loops against a single-user INT4 GPU baseline; against optimized GPU serving (batched vLLM, CPU+NVMe park, power-gate) the robust advantage shrinks to 1.36-4.69x and remains >=41x on parked sessions with multi-hour residency.