HPVM-HDC: A Heterogeneous Programming System for Accelerating Hyperdimensional Computing

2024-10-19

AI Summary

Existing Hyperdimensional Computing (HDC) implementations rely on manual, low-level programming in CPU/GPU/FPGA-specific languages, resulting in inefficient cross-platform development and deployment across diverse hardware—including CPUs, GPUs, FPGAs, digital ASICs, and resistive RAM (ReRAM)—and lacking a unified, high-efficiency programming system. Method: This paper introduces HDC++, a unified high-level programming language, and HPVM-HDC, a heterogeneous compilation framework. It pioneers compiler optimizations for automatic binarization and reduction-based puncturing, enabling end-to-end compilation and cycle-accurate simulation of HDC applications on both digital ASICs and ReRAM-based accelerators. Contribution/Results: Experiments show HDC++ achieves 1.17× average speedup over optimized CUDA baselines on CPUs and GPUs, with 1.6× reduction in source code volume. Crucially, it establishes the first full-stack compilation pipeline from high-level HDC abstractions to emerging compute-in-memory hardware (ReRAM), laying foundational support for co-design of HDC software and hardware.

Technology Category

• Machine Learning: Hardware-aware ML
• Humans and AI: Other Foundations of Human Computation & AI
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Hyperdimensional Computing (HDC), a technique inspired by cognitive models of computation, has been proposed as an efficient and robust alternative basis for machine learning. HDC programs are often manually written in low-level and target specific languages targeting CPUs, GPUs, and FPGAs—these codes cannot be easily retargeted onto HDC-specific accelerators. No previous programming system enables productive development of HDC programs and generates efficient code for several hardware targets. We propose a heterogeneous programming system for HDC: a novel programming language, HDC++, for writing applications using a unified programming model, including HDC-specific primitives to improve programmability, and a heterogeneous compiler, HPVM-HDC, that provides an intermediate representation for compiling HDC programs to many hardware targets. We implement two tuning optimizations, automatic binarization and reduction perforation, that exploit the error resilient nature of HDC. Our evaluation shows that HPVM-HDC generates performance-competitive code for CPUs and GPUs, achieving a geomean speed-up of 1.17x over optimized baseline CUDA implementations with a geomean reduction in total lines of code of 1.6x across CPUs and GPUs. Additionally, HPVM-HDC targets an HDC Digital ASIC and an HDC ReRAM accelerator simulator, enabling the first execution of HDC applications on these devices.

Problem

• Lack of portable programming system for Hyperdimensional Computing applications
• Difficulty in retargeting HDC code across CPUs, GPUs, and FPGAs
• Absence of optimized compilation for HDC-specific hardware accelerators

Innovation

• HDC++ language for unified HDC programming
• HPVM-HDC compiler for multiple hardware targets
• Automatic binarization and reduction perforation optimizations