Tuning Algorithmic and Architectural Hyperparameters in Graph-Based Semi-Supervised Learning with Provable Guarantees

2025-02-18

AI Summary

This paper addresses the lack of theoretical guarantees for hyperparameter tuning in graph-structured semi-supervised learning, proposing the first provably efficient hyperparameter selection framework. Methodologically: (1) it establishes the first tight bound—$O(log n)$—on the pseudo-dimension of the hyperparameter space, yielding a rigorous upper bound on sample complexity; (2) it designs an interpolatable GCN-GAT hybrid architecture, providing Rademacher complexity guarantees for self-loop weights and interpolation coefficients; and (3) it unifies the generalization analysis of label propagation methods, SGC, and graph neural networks. The key contribution is the first tight theoretical characterization of hyperparameter tuning in graph learning—balancing provable correctness with empirical effectiveness—and thereby advancing the development of trustworthy graph learning frameworks.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Graph-based semi-supervised learning is a powerful paradigm in machine learning for modeling and exploiting the underlying graph structure that captures the relationship between labeled and unlabeled data. A large number of classical as well as modern deep learning based algorithms have been proposed for this problem, often having tunable hyperparameters. We initiate a formal study of tuning algorithm hyperparameters from parameterized algorithm families for this problem. We obtain novel $O(log n)$ pseudo-dimension upper bounds for hyperparameter selection in three classical label propagation-based algorithm families, where $n$ is the number of nodes, implying bounds on the amount of data needed for learning provably good parameters. We further provide matching $Omega(log n)$ pseudo-dimension lower bounds, thus asymptotically characterizing the learning-theoretic complexity of the parameter tuning problem. We extend our study to selecting architectural hyperparameters in modern graph neural networks. We bound the Rademacher complexity for tuning the self-loop weighting in recently proposed Simplified Graph Convolution (SGC) networks. We further propose a tunable architecture that interpolates graph convolutional neural networks (GCN) and graph attention networks (GAT) in every layer, and provide Rademacher complexity bounds for tuning the interpolation coefficient.

Problem

• Tuning hyperparameters in graph-based learning
• Provable guarantees for parameter selection
• Complexity analysis of hyperparameter tuning

Innovation

• Tuning hyperparameters in graph learning
• Bounding Rademacher complexity for SGC
• Interpolating GCN and GAT architectures