RVANNS: Mixed-Precision Indexing and Locality-Aware Graph Traversal on RISC-V

2026-08-09

AI Summary

This work addresses the high memory overhead and poor cache locality that hinder CPU-based approximate nearest neighbor search (ANNS). Targeting the RISC-V Vector extension (RVV), it proposes a hybrid-precision multilayer index combined with an ROrder graph node reordering strategy, which integrates vector reconstruction and distance computation to transform irregular memory accesses into forward-dense streams. The approach employs 8-bit affine bases with sparse FP16/FP32 residual representations, adjacency list remapping, and LMUL-based register grouping optimizations. Evaluated on 128-bit and 256-bit RVV processors, the design achieves speedups of 3.39× and 4.94×, respectively, delivering 2.27–2.76× higher throughput than RVV SIMD with FP32 precision and demonstrating 1.82–2.27× better energy efficiency compared to AVX-512, SVE, and GPU baselines.

Abstract

Approximate nearest neighbor search (ANNS) on CPUs is increasingly constrained by candidate-vector movement and decoding rather than peak arithmetic throughput. Although the RISC-V Vector Extension (RVV) provides vector-length-agnostic execution and LMUL-based register grouping, generic low-precision decoding still incurs conversion overhead, while irregular graph traversal generates scattered accesses that degrade cache locality and memory-level parallelism. We present RVANNS, an RVV-oriented ANNS engine that jointly optimizes vector representation and graph locality. Its Mixed-Precision Multi-Layer Index (MPMI) represents each vector with a dense 8-bit affine base and sparse FP16/FP32 residuals, fusing reconstruction with distance accumulation and aligning widening with LMUL-sized register groups. ROrder co-locates likely co-visited graph nodes and sorts remapped adjacency lists, transforming scattered payload probes into denser, predominantly forward-moving address streams. Integrated into Milvus, RVANNS achieves 3.39x and 4.94x speedups over scalar execution on real 128-bit and 256-bit RVV processors, respectively. Under controlled HNSW configurations, it improves throughput by 2.27--2.76x over RVV SIMD+FP32 and by 1.18--1.59x over the corresponding AVX-512 and SVE baselines. On Cohere10M, it further delivers 1.82--2.27x higher QPS/W than the evaluated GPU baselines.

Problem

• Approximate Nearest Neighbor Search
• RISC-V Vector Extension
• Memory Locality
• Low-Precision Decoding
• Irregular Graph Traversal

Innovation

• Mixed-Precision Indexing
• Locality-Aware Graph Traversal
• RISC-V Vector Extension
• Approximate Nearest Neighbor Search
• Memory Access Optimization