🤖 AI Summary
To address the high latency and energy consumption of Tensor Train Decomposition (TTD) on edge devices—caused by repeated SVD computations and matrix multiplications—this paper proposes TT-Edge, a hardware-software co-design framework. Methodologically, it decouples SVD into two stages—bidiagonalization and diagonalization—enabling reuse of the GEMM accelerator and integrating a shared floating-point unit to minimize data movement overhead. A dedicated TTD engine is designed atop the RISC-V architecture and tightly coupled with the GEMM module, validated on both FPGA prototypes and 45 nm backend synthesis. Experimental results on compressing ResNet-32 demonstrate a 1.7× speedup over a pure GEMM-based baseline, a 40.2% improvement in system-level energy efficiency, and only a 4% increase in hardware area. TT-Edge thus achieves efficient, low-power, and area-efficient TTD acceleration for edge deployment.
📝 Abstract
The growing demands of distributed learning on resource constrained edge devices underscore the importance of efficient on device model compression. Tensor Train Decomposition (TTD) offers high compression ratios with minimal accuracy loss, yet repeated singular value decompositions (SVDs) and matrix multiplications can impose significant latency and energy costs on low power processors. In this work, we present TT-Edge, a hardware software co designed framework aimed at overcoming these challenges. By splitting SVD into two phases--bidiagonalization and diagonalization--TT-Edge offloads the most compute intensive tasks to a specialized TTD Engine. This engine integrates tightly with an existing GEMM accelerator, thereby curtailing the frequent matrix vector transfers that often undermine system performance and energy efficiency. Implemented on a RISC-V-based edge AI processor, TT-Edge achieves a 1.7x speedup compared to a GEMM only baseline when compressing a ResNet 32 model via TTD, while reducing overall energy usage by 40.2 percent. These gains come with only a 4 percent increase in total power and minimal hardware overhead, enabled by a lightweight design that reuses GEMM resources and employs a shared floating point unit. Our experimental results on both FPGA prototypes and post-synthesis power analysis at 45 nm demonstrate that TT-Edge effectively addresses the latency and energy bottlenecks of TTD based compression in edge environments.