CORTEX: High-Quality Cross-Domain Organization of Web-Scale Corpora through Ontological Corpus Graph

2026-06-29

AI Summary

Existing corpus construction methods produce only flat document collections, lacking systematic knowledge organization and thus failing to meet the demand of large language models for high-quality structured data. This work proposes the CORTEX framework, which introduces the first three-layer heterogeneous Ontology-based Corpus Graph (OCG), comprising a quality-optimized content layer, a lightweight ontology layer evolved via LLM-driven mechanisms, and a cross-domain alignment layer. This architecture enables cross-domain associations at arbitrary granularities and supports automatic ontology evolution. Leveraging this framework, we construct a structured, high-quality corpus of 24.14 billion tokens and release CortexBench, a benchmark for cross-domain retrieval and reasoning, demonstrating its effectiveness across eight state-of-the-art large language models.

Abstract

The continuous evolution of large language models drives escalating demands on data scale and quality, and as different training stages impose increasingly tailored data requirements, systematic organization of high-quality corpora becomes indispensable. Existing corpus construction pipelines confine the resulting corpora to flat, undifferentiated document collections, universally lacking systematic knowledge organization. We present Cortex, to our knowledge the first framework that elevates web-scale corpus construction from flat document filtering to structured knowledge organization through an Ontological Corpus Graph (OCG), a three-layer heterogeneous structure unifying a quality-refined content layer, a hierarchical lightweight ontology layer via LLM-driven automated evolution, and a cross-domain alignment layer enabling inter-domain association at arbitrary taxonomic resolution. Comprehensive experiments confirm the effectiveness of Cortex. In particular, we leverage the OCG to synthesize CortexBench, a cross-domain search-and-reasoning benchmark whose evaluation across eight frontier LLMs validates the effectiveness of quality refinement, domain organization, and cross-domain data synthesis. We will publicly release the complete codebase, a 24.14B-token refined corpus with its OCG, and CortexBench.

Problem

• corpus organization
• knowledge structure
• cross-domain alignment
• large language models
• ontological graph

Innovation

• Ontological Corpus Graph
• corpus organization
• cross-domain alignment
• LLM-driven ontology
• structured knowledge