Unexpected Knowledge: Auditing Wikipedia and Grokipedia Search Recommendations

2025-12-18

AI Summary

This study addresses the lack of systematic comparison between human-curated and AI-generated encyclopedic search recommendation mechanisms, focusing on semantic alignment, content overlap, and user exploration path divergence. Method: We constructed a query set of nearly 10,000 neutral English terms and their substrings, collected over 70,000 search results from Wikipedia and the AI-generated encyclopedia Grokipedia, and applied multimodal analysis—including semantic similarity computation (using Sentence-BERT), expert-driven topical annotation, cross-platform overlap quantification, and multi-step path evolution modeling. Contribution/Results: Both platforms exhibit pervasive “serendipitous knowledge” — low inter-platform result overlap (<32%) and suboptimal semantic relevance for identical queries. Grokipedia shows statistically significant thematic skew toward culture and fiction, revealing structural divergence in knowledge organization and retrieval logic. The study establishes a novel methodology and empirical benchmark for evaluating the reliability and epistemic coherence of generative knowledge systems.

Technology Category

• Data Mining & Knowledge Management: Knowledge Acquisition from the Web
• Humans and AI: Crowd Sourcing and Human Computation
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

Encyclopedic knowledge platforms are key gateways through which users explore information online. The recent release of Grokipedia, a fully AI-generated encyclopedia, introduces a new alternative to traditional, well-established platforms like Wikipedia. In this context, search engine mechanisms play an important role in guiding users exploratory paths, yet their behavior across different encyclopedic systems remains underexplored. In this work, we address this gap by providing the first comparative analysis of search engine in Wikipedia and Grokipedia. Using nearly 10,000 neutral English words and their substrings as queries, we collect over 70,000 search engine results and examine their semantic alignment, overlap, and topical structure. We find that both platforms frequently generate results that are weakly related to the original query and, in many cases, surface unexpected content starting from innocuous queries. Despite these shared properties, the two systems often produce substantially different recommendation sets for the same query. Through topical annotation and trajectory analysis, we further identify systematic differences in how content categories are surfaced and how search engine results evolve over multiple stages of exploration. Overall, our findings show that unexpected search engine outcomes are a common feature of both the platforms, even though they exhibit discrepancies in terms of topical distribution and query suggestions.

Problem

• Compare search engine behaviors between Wikipedia and Grokipedia
• Analyze semantic alignment and overlap in search recommendations
• Identify differences in content categories and exploration trajectories

Innovation

• Comparative analysis of Wikipedia and Grokipedia search engines
• Using 10,000 neutral words and substrings as queries
• Examining semantic alignment, overlap, and topical structure