Triples and Knowledge-Infused Embeddings for Clustering and Classification of Scientific Documents

2025-12-19

arXiv.org

AI Summary

This study investigates how structured knowledge, such as knowledge triples, can be effectively leveraged to enhance clustering and classification performance on scientific literature. We design a modular pipeline to systematically evaluate four document representation strategies—abstract-only, triples-only, fused, and hybrid—across multiple Transformer-based embeddings (MiniLM, MPNet, SciBERT, SPECTER) and clustering algorithms (KMeans, GMM, HDBSCAN). Experimental results demonstrate that using abstracts alone achieves the best classification performance, with both accuracy and macro F1 score reaching 0.923. In contrast, naively integrating knowledge triples does not consistently improve performance and yields highly configuration-dependent outcomes. Our work establishes a reproducible benchmark and offers practical guidance for knowledge-enhanced representation of scientific documents.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Machine Learning: Representation Learning

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

The increasing volume and complexity of scientific literature demand robust methods for organizing and understanding research documents. In this study, we explore how structured knowledge, specifically, subject-predicate-object triples, can enhance the clustering and classification of scientific papers. We propose a modular pipeline that combines unsupervised clustering and supervised classification over multiple document representations: raw abstracts, extracted triples, and hybrid formats that integrate both. Using a filtered arXiv corpus, we extract relational triples from abstracts and construct four text representations, which we embed using four state-of-the-art transformer models: MiniLM, MPNet, SciBERT, and SPECTER. We evaluate the resulting embeddings with KMeans, GMM, and HDBSCAN for unsupervised clustering, and fine-tune classification models for arXiv subject prediction. Our results show that full abstract text yields the most coherent clusters, but that hybrid representations incorporating triples consistently improve classification performance, reaching up to 92.6% accuracy and 0.925 macro-F1. We also find that lightweight sentence encoders (MiniLM, MPNet) outperform domain-specific models (SciBERT, SPECTER) in clustering, while SciBERT excels in structured-input classification. These findings highlight the complementary benefits of combining unstructured text with structured knowledge, offering new insights into knowledge-infused representations for semantic organization of scientific documents.

Problem

• scientific document classification
• knowledge infusion
• triples
• document clustering
• structured knowledge

Innovation

• knowledge-infused embeddings
• scientific document clustering
• triple extraction
• transformer embeddings
• reproducible benchmark