LoRAP: Low-Rank Aggregation Prompting for Quantized Graph Neural Networks Training

2026-01-21

AI Summary

This work addresses the significant performance degradation of graph neural networks (GNNs) under low-bit quantization, primarily caused by information loss during the feature aggregation phase. To mitigate this issue, the authors propose Low-Rank Aggregation Prompting (LoRAP), a novel approach that introduces prompt learning into quantization-aware training (QAT) for GNNs. LoRAP injects lightweight, input-dependent, low-rank learnable prompts during the message-passing aggregation stage to correct quantization errors. Notably, it preserves the original model backbone and incurs minimal computational overhead. Extensive experiments across nine graph datasets and four mainstream QAT frameworks demonstrate that LoRAP consistently enhances the performance of low-bit GNNs, substantially outperforming existing state-of-the-art methods.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Natural Language Processing: Prompt Engineering / Prompting
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Graph Neural Networks (GNNs) are neural networks that aim to process graph data, capturing the relationships and interactions between nodes using the message-passing mechanism. GNN quantization has emerged as a promising approach for reducing model size and accelerating inference in resource-constrained environments. Compared to quantization in LLMs, quantizing graph features is more emphasized in GNNs. Inspired by the above, we propose to leverage prompt learning, which manipulates the input data, to improve the performance of quantization-aware training (QAT) for GNNs. To mitigate the issue that prompting the node features alone can only make part of the quantized aggregation result optimal, we introduce Low-Rank Aggregation Prompting (LoRAP), which injects lightweight, input-dependent prompts into each aggregated feature to optimize the results of quantized aggregations. Extensive evaluations on 4 leading QAT frameworks over 9 graph datasets demonstrate that LoRAP consistently enhances the performance of low-bit quantized GNNs while introducing a minimal computational overhead.

Problem

• Graph Neural Networks
• Quantization
• Prompt Learning
• Aggregation
• Quantization-Aware Training

Innovation

• Low-Rank Prompting
• Graph Neural Networks
• Quantization-Aware Training
• Aggregation Optimization
• Prompt Learning