CIMERA: Compute-in-Interconnect and Memory with Reconfigurable Precision for LLM Inference

2026-07-15

AI Summary

This work addresses the challenge of deploying large language models on energy-constrained devices, where high computational and memory demands severely limit efficiency. To overcome this, the authors propose an integrated compute-interconnect-memory architecture that uniquely combines in-interconnect computing with a reconfigurable precision mechanism. By employing precision-aware scheduling, the system dynamically adapts to model heterogeneity while preserving accuracy. Experimental results demonstrate substantial improvements in throughput and energy efficiency: compared to the NVIDIA H100 GPU, the proposed approach achieves up to 25× and 10× higher energy efficiency for 1B and 13B parameter models, respectively.

Abstract

LLM impose significant computational and memory demands, creating challenges for energy-efficient inference across platforms ranging from data centers to power-constrained edge devices. Weight precision plays a critical role in balancing inference accuracy, throughput, and energy consumption, while modern LLM workloads exhibit pronounced heterogeneity and tolerance that favors adaptive precision execution. This paper presents CIMERA, a reconfigurable-precision LLM inference accelerator that integrates compute-in-interconnect and memory to mitigate the memory wall and enable precision-aware execution. Compared to Nvidia H100, CIMERA delivers up to $25\times$ and $10\times$ higher energy efficiency for 1B and 13B models, respectively.

Problem

• LLM inference
• energy efficiency
• weight precision
• memory wall
• heterogeneous workloads

Innovation

• reconfigurable precision
• compute-in-interconnect
• compute-in-memory
• LLM inference
• energy efficiency