ContextGNN goes to Elliot: Towards Benchmarking Relational Deep Learning for Static Link Prediction (aka Personalized Item Recommendation)

2025-03-20

AI Summary

This study evaluates and enhances ContextGNN’s performance on static link prediction—specifically, personalized item recommendation. We first integrate ContextGNN into the Elliot unified evaluation framework and conduct systematic benchmarking against six state-of-the-art GNN-based recommenders on three standard datasets (e.g., Amazon-Books). Leveraging relational deep learning via message passing, we empirically reveal ContextGNN’s representational limitations in static settings, where it underperforms several advanced GNN methods. Our key contributions are: (1) establishing the first reproducible ContextGNN benchmark within Elliot; (2) empirically characterizing its performance ceiling on static recommendation tasks; and (3) open-sourcing all code and experimental configurations to facilitate standardized, fair evaluation of relational recommendation models. This work advances methodological rigor and transparency in graph-based recommender system research.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Relational deep learning (RDL) settles among the most exciting advances in machine learning for relational databases, leveraging the representational power of message passing graph neural networks (GNNs) to derive useful knowledge and run predicting tasks on tables connected through primary-to-foreign key links. The RDL paradigm has been successfully applied to recommendation lately, through its most recent representative deep learning architecture namely, ContextGNN. While acknowledging ContextGNN's improved performance on real-world recommendation datasets and tasks, preliminary tests for the more traditional static link prediction task (aka personalized item recommendation) on the popular Amazon Book dataset have demonstrated how ContextGNN has still room for improvement compared to other state-of-the-art GNN-based recommender systems. To this end, with this paper, we integrate ContextGNN within Elliot, a popular framework for reproducibility and benchmarking analyses, counting around 50 state-of-the-art recommendation models from the literature to date. On such basis, we run preliminary experiments on three standard recommendation datasets and against six state-of-the-art GNN-based recommender systems, confirming similar trends to those observed by the authors in their original paper. The code is publicly available on GitHub: https://github.com/danielemalitesta/Rel-DeepLearning-RecSys.

Problem

• Benchmarking ContextGNN for static link prediction
• Comparing ContextGNN with GNN-based recommender systems
• Evaluating performance on standard recommendation datasets

Innovation

• Leverages message passing GNNs for relational learning
• Integrates ContextGNN with Elliot framework
• Benchmarks against six GNN-based recommender systems