VecIntrinBench: Benchmarking Cross-Architecture Intrinsic Code Migration for RISC-V Vector

2025-11-24

AI Summary

Existing benchmarks lack systematic evaluation of large language models’ (LLMs) capability to migrate intrinsic functions across architectures, particularly for the RISC-V Vector (RVV) extension. Method: We introduce VecIntrinBench—the first cross-architecture intrinsic migration benchmark—comprising 50 open-source computational tasks, each implemented in scalar, RVV, ARM NEON, and x86 SIMD variants, accompanied by comprehensive functional correctness and performance test suites. Our methodology integrates rule-based translation with LLM-driven code generation and automated verification. Contribution/Results: This work presents the first systematic assessment of LLMs for RVV intrinsic migration, demonstrating that LLM-generated implementations achieve functional correctness and performance on par with—or exceeding—traditional rule-based mapping. The fully open-sourced VecIntrinBench serves as foundational infrastructure for LLM-augmented heterogeneous architecture migration research and advances the RISC-V software ecosystem with empirical validation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Intrinsic functions are specialized functions provided by the compiler that efficiently operate on architecture-specific hardware, allowing programmers to write optimized code in a high-level language that fully exploits hardware features. Using intrinsics to vectorize core code blocks is a standard optimization method in high-performance libraries, often requiring specific vector optimization implementations for multiple mainstream architectures. The promising RISC-V software ecosystem has a significant demand for algorithm library migration and adaptation. Translating existing intrinsic functions to RISC-V Vector (RVV) intrinsic functions across architectures is currently a mainstream approach. Rule-based intrinsic mapping methods and LLM-based code generation can help developers address the code migration challenge. However, existing intrinsic code benchmarks focus on mainstream SIMD intrinsics and lack support for the emerging RISC-V architecture. There is currently no benchmark that comprehensively evaluates the intrinsic migration capabilities for the RVV extension. To fill this gap, we propose VecIntrinBench, the first intrinsic benchmark encompassing RVV extensions. It includes 50 function-level tasks from open source repositories, implemented as scalars, RVV intrinsics, Arm Neon intrinsics, and x86 intrinsics, along with comprehensive functional and performance test cases. We systematically evaluated various code migration approaches on VecIntrinBench, yielding a series of insightful findings. The results demonstrate that advanced Large Language Models (LLMs) achieve a similar effect as rule-based mapping approaches for RISC-V code migration, while also delivering superior performance. We further analyze the reasons and identify future directions for LLM development in the code migration field. The VecIntrinBench is open-sourced to benefit the broader community and developers.

Problem

• Lack of benchmarks for RISC-V Vector intrinsic code migration evaluation
• Need to assess cross-architecture intrinsic translation capabilities for RVV
• Evaluate performance of rule-based mapping versus LLM approaches for migration

Innovation

• Benchmarking RISC-V Vector intrinsic migration across architectures
• Evaluating rule-based and LLM approaches for code migration
• Providing comprehensive test cases for RVV extension evaluation