Selected work

Representative Papers

GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning

May 26, 2025

LoRA suffers from structural bottlenecks at high ranks, causing gradient entanglement across input channels, which leads to overfitting and performance saturation—hindering its ability to approximate full fine-tuning (FFT). To address this, we propose Granular Low-Rank Adaptation (GraLoRA), the first sub-block-level low-rank adaptation framework: it partitions the weight matrix into fine-grained blocks and assigns each block an independent low-rank adapter, explicitly decoupling gradient propagation paths. This design incurs virtually zero additional parameters or computational overhead while substantially enhancing representational capacity and FFT approximation fidelity. On HumanEval+, GraLoRA achieves a +8.5% improvement in Pass@1, consistently outperforming LoRA and other PEFT baselines across diverse model scales and rank configurations. Its performance demonstrates strong robustness and scalability.

0 citationsRead paper

Debunking the CUDA Myth Towards GPU-based AI Systems

Dec 31, 2024

This study systematically evaluates the feasibility of replacing NVIDIA A100 GPUs with Intel Gaudi-2 NPUs for AI inference serving. Method: We design a microbenchmarking framework assessing four critical dimensions—compute performance, memory bandwidth, inter-device communication efficiency, and energy efficiency—and conduct end-to-end AI workload comparisons. We further perform deep software co-optimization, including FBGEMM operator customization and vLLM inference engine adaptation tailored to Gaudi-2’s NPU architecture. Contribution/Results: To our knowledge, this is the first real-world inference evaluation demonstrating that Gaudi-2 achieves A100-level throughput and energy efficiency across mainstream LLMs—reaching up to 92% of A100’s throughput while delivering 1.3× higher performance-per-watt. We propose NPU-aware operator fusion and scheduling optimizations that substantially alleviate software-stack immaturity bottlenecks. Our findings indicate that Gaudi-2 possesses tangible technical potential to challenge GPU dominance in inference, contingent upon framework-level, hardware-software co-design.

0 citationsRead paper

Recent publications

Latest Papers

GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning

May 26, 2025

LoRA suffers from structural bottlenecks at high ranks, causing gradient entanglement across input channels, which leads to overfitting and performance saturation—hindering its ability to approximate full fine-tuning (FFT). To address this, we propose Granular Low-Rank Adaptation (GraLoRA), the first sub-block-level low-rank adaptation framework: it partitions the weight matrix into fine-grained blocks and assigns each block an independent low-rank adapter, explicitly decoupling gradient propagation paths. This design incurs virtually zero additional parameters or computational overhead while substantially enhancing representational capacity and FFT approximation fidelity. On HumanEval+, GraLoRA achieves a +8.5% improvement in Pass@1, consistently outperforming LoRA and other PEFT baselines across diverse model scales and rank configurations. Its performance demonstrates strong robustness and scalability.

0 citationsRead paper

Debunking the CUDA Myth Towards GPU-based AI Systems

Dec 31, 2024

This study systematically evaluates the feasibility of replacing NVIDIA A100 GPUs with Intel Gaudi-2 NPUs for AI inference serving. Method: We design a microbenchmarking framework assessing four critical dimensions—compute performance, memory bandwidth, inter-device communication efficiency, and energy efficiency—and conduct end-to-end AI workload comparisons. We further perform deep software co-optimization, including FBGEMM operator customization and vLLM inference engine adaptation tailored to Gaudi-2’s NPU architecture. Contribution/Results: To our knowledge, this is the first real-world inference evaluation demonstrating that Gaudi-2 achieves A100-level throughput and energy efficiency across mainstream LLMs—reaching up to 92% of A100’s throughput while delivering 1.3× higher performance-per-watt. We propose NPU-aware operator fusion and scheduling optimizations that substantially alleviate software-stack immaturity bottlenecks. Our findings indicate that Gaudi-2 possesses tangible technical potential to challenge GPU dominance in inference, contingent upon framework-level, hardware-software co-design.

0 citationsRead paper