Evaluation and Explainability of Unsupervised Scholarly Collaboration Recommendations

2026-07-05

AI Summary

This study investigates unsupervised research collaboration recommendation based on academic texts and its interpretability. By constructing a deduplicated dataset of scholarly publications and using historical co-authorship as a proxy ground truth, the authors propose a constrained evaluation setting that partially removes publication overlap between candidate collaborators. They systematically evaluate three approaches: TF-IDF, LDA, and SciBERT embeddings (retrieved via Faiss), integrating intrinsic topic-based explanations with post-hoc explanations generated by large language models. Results show that TF-IDF achieves the best performance on complete data but exhibits poor robustness, whereas topic modeling and embedding-based methods demonstrate greater stability under missing information, offering a favorable trade-off between distributional semantic generalization and interpretability.

Abstract

In this paper, we examine unsupervised, content-based collaboration recommendations using publication text in scholarly settings. We compare three families of methods: a TF-IDF baseline, topic-based models (LDA and BERTopic, including clone variants), and embedding-based retrieval using SciBERT with Faiss. To evaluate model behavior beyond simple lexical matching, we introduce a constrained setting where publication overlap between researchers is partially removed while still using historical co-authorship as proxy ground truth for post-hoc evaluation. Results show clear differences across methods. TF-IDF performs best under full information but drops significantly as overlap is reduced. In contrast, topic-based and embedding-based approaches show more stable performance, suggesting they capture broader distributional similarities, rather than relying only on direct lexical overlap. We also examine explainability through two perspectives: intrinsic topic-based explanations and post-hoc, retrieval-based explanations generated using language models. These provide complementary trade-offs between transparency and human readability.

Problem

• unsupervised collaboration recommendation
• scholarly recommendation
• explainability
• publication text
• evaluation

Innovation

• unsupervised collaboration recommendation
• constrained evaluation setting
• distributional similarity
• explainable AI
• embedding-based retrieval