VecTrans: LLM Transformation Framework for Better Auto-vectorization on High-performance CPU

2025-03-25

AI Summary

This work addresses the limited applicability of large language models (LLMs) in compiler auto-vectorization due to hallucination and insufficient domain knowledge. We propose the first LLM-driven code pattern rewriting framework. Methodologically, it integrates compiler-based static analysis to identify non-vectorizable code segments, employs prompt engineering to guide LLMs in performing semantics-preserving, IR-level code restructuring, and enforces correctness via IR-level equivalence verification. Our key contribution is bridging LLMs’ pattern recognition capability with compilers’ semantic precision. Evaluated on the TSVC benchmark, the framework successfully auto-vectorizes 23 out of 50 functions—previously unvectorizable by mainstream compilers including Clang, GCC, and BiSheng—achieving a 46% success rate and an average speedup of 2.02×.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large language models (LLMs) have demonstrated great capabilities in code generation, yet their effective application in compiler optimizations remains an open challenge due to issues such as hallucinations and a lack of domain-specific reasoning. Vectorization, a crucial optimization for enhancing code performance, often fails because of the compiler's inability to recognize complex code patterns, which commonly require extensive empirical expertise. LLMs, with their ability to capture intricate patterns, thus providing a promising solution to this challenge. This paper presents VecTrans, a novel framework that leverages LLMs to enhance compiler-based code vectorization. VecTrans first employs compiler analysis to identify potentially vectorizable code regions. It then utilizes an LLM to refactor these regions into patterns that are more amenable to the compiler's auto-vectorization. To ensure semantic correctness, VecTrans further integrates a hybrid validation mechanism at the intermediate representation (IR) level. With the above efforts, VecTrans combines the adaptability of LLMs with the precision of compiler vectorization, thereby effectively opening up the vectorization opportunities. Experimental results show that among all 50 TSVC functions unvectorizable by Clang, GCC, and BiShengCompiler, VecTrans successfully vectorizes 23 cases (46%) and achieves an average speedup of 2.02x, greatly surpassing state-of-the-art performance.

Problem

• Enhancing compiler-based code vectorization using LLMs
• Overcoming compiler limitations in recognizing complex code patterns
• Ensuring semantic correctness in LLM-refactored vectorized code

Innovation

• LLM refactors code for better vectorization
• Hybrid validation ensures semantic correctness
• Compiler analysis identifies vectorizable regions