Efficient Architecture for RISC-V Vector Memory Access

2025-04-11

AI Summary

RISC-V vector processors suffer from efficiency bottlenecks in strided (gather/scatter) and segmented memory accesses: the former incurs high overhead from crossbar networks and large buffers, while the latter requires costly row-column transposition, degrading performance or inflating area. This paper proposes EARTH, a novel architecture introducing shift-based memory access optimization. It replaces conventional crossbars with a dedicated shift network for low-overhead strided access and employs shift register files to enable bufferless, in-place batch transposition—unifying hardware optimization for both challenging access patterns for the first time. Additionally, it integrates a segmented-aware register file and cache-line-level memory access aggregation. FPGA evaluation demonstrates 4–8× speedup on strided-intensive benchmarks, with 9% area reduction and 41% power savings. EARTH has been validated at silicon level on an open-source RISC-V vector core.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Distributed Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

Vector processors frequently suffer from inefficient memory accesses, particularly for strided and segment patterns. While coalescing strided accesses is a natural solution, effectively gathering or scattering elements at fixed strides remains challenging. Naive approaches rely on high-overhead crossbars that remap any byte between memory and registers, leading to physical design issues. Segment operations require row-column transpositions, typically handled using either element-level in-place transposition (degrading performance) or large buffer-based bulk transposition (incurring high area overhead). In this paper, we present EARTH, a novel vector memory access architecture designed to overcome these challenges through shifting-based optimizations. For strided accesses, EARTH integrates specialized shift networks for gathering and scattering elements. After coalescing multiple accesses within the same cache line, data is routed between memory and registers through the shifting network with minimal overhead. For segment operations, EARTH employs a shifted register bank enabling direct column-wise access, eliminating dedicated segment buffers while providing high-performance, in-place bulk transposition. Implemented on FPGA with Chisel HDL based on an open-source RISC-V vector unit, EARTH enhances performance for strided memory accesses, achieving 4x-8x speedups in benchmarks dominated by strided operations. Compared to conventional designs, EARTH reduces hardware area by 9% and power consumption by 41%, significantly advancing both performance and efficiency of vector processors.

Problem

• Improves inefficient strided memory access in vector processors
• Reduces overhead in gathering/scattering elements at fixed strides
• Eliminates dedicated buffers for high-performance segment operations

Innovation

• Specialized shift networks for strided accesses
• Shifted register bank for segment operations
• Coalescing and shifting reduce overhead