CPT: Competence-progressive Training Strategy for Few-shot Node Classification

2024-02-01

arXiv.org

AI Summary

To address few-shot node classification under extreme label scarcity—particularly when certain classes exhibit severely limited labeled instances—this paper proposes a capability-progressive meta-learning framework. Methodologically, it introduces a two-stage curriculum learning paradigm: the first stage trains foundational discriminative capabilities, while the second stage dynamically schedules task difficulty based on real-time performance evaluation of the meta-learner, enabling cognition-informed adaptive training. The framework integrates graph neural networks (GNNs), model-agnostic meta-learning (MAML) or prototypical networks (ProtoNet), quantitative task-difficulty modeling, and a dynamic curriculum scheduling algorithm. Its key innovation lies in the first integration of meta-learner capability assessment into the curriculum scheduling mechanism, overcoming the limitations of conventional random task sampling. Extensive experiments on benchmark citation networks—including Cora, CiteSeer, and PubMed—demonstrate that the proposed method achieves average accuracy improvements of 3.2–5.8 percentage points over state-of-the-art few-shot GNN approaches.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Graph Neural Networks (GNNs) have made significant advancements in node classification, but their success relies on sufficient labeled nodes per class in the training data. Real-world graph data often exhibits a long-tail distribution with sparse labels, emphasizing the importance of GNNs' ability in few-shot node classification, which entails categorizing nodes with limited data. Traditional episodic meta-learning approaches have shown promise in this domain, but they face an inherent limitation: it might lead the model to converge to suboptimal solutions because of random and uniform task assignment, ignoring task difficulty levels. This could lead the meta-learner to face complex tasks too soon, hindering proper learning. Ideally, the meta-learner should start with simple concepts and advance to more complex ones, like human learning. So, we introduce CPT, a novel two-stage curriculum learning method that aligns task difficulty with the meta-learner's progressive competence, enhancing overall performance. Specifically, in CPT's initial stage, the focus is on simpler tasks, fostering foundational skills for engaging with complex tasks later. Importantly, the second stage dynamically adjusts task difficulty based on the meta-learner's growing competence, aiming for optimal knowledge acquisition. Extensive experiments on popular node classification datasets demonstrate significant improvements of our strategy over existing methods.

Problem

• Graph Neural Networks
• Limited Labeled Data
• Node Classification Accuracy

Innovation

• CPT
• Graph Neural Networks
• Meta-Learning