MajinBook: An open catalogue of digital world literature with likes

2025-11-14

AI Summary

Traditional digital corpora (e.g., HathiTrust) suffer from geographic, linguistic, and publishing biases. To address this, we propose an open corpus construction methodology integrating shadow libraries (Library Genesis/Z-Library) with structured Goodreads data. Our approach prioritizes native EPUB files and leverages crowdsourced metadata alongside social book reviews; it employs a high-precision cross-source matching algorithm designed to ensure legal compliance and multilingual scalability. We release the first open, decentralized, and reproducible English-language corpus spanning three centuries—comprising 539,000 distinct works—with granular annotations including original publication year, genre, and multidimensional popularity metrics. A rigorous empirical evaluation confirms high link accuracy. This resource advances computational social science and cultural analytics by providing a high-quality, ethically sourced, and methodologically transparent foundational corpus.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Humans and AI: Crowd Sourcing and Human Computation
• Application Domains: Humanities & Computational Social Science

Application Category

• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

This data paper introduces MajinBook, an open catalogue designed to facilitate the use of shadow libraries--such as Library Genesis and Z-Library--for computational social science and cultural analytics. By linking metadata from these vast, crowd-sourced archives with structured bibliographic data from Goodreads, we create a high-precision corpus of over 539,000 references to English-language books spanning three centuries, enriched with first publication dates, genres, and popularity metrics like ratings and reviews. Our methodology prioritizes natively digital EPUB files to ensure machine-readable quality, while addressing biases in traditional corpora like HathiTrust, and includes secondary datasets for French, German, and Spanish. We evaluate the linkage strategy for accuracy, release all underlying data openly, and discuss the project's legal permissibility under EU and US frameworks for text and data mining in research.

Problem

• Creating an open catalogue for shadow libraries in computational research
• Linking metadata from archives with structured bibliographic data sources
• Addressing biases in traditional corpora while ensuring legal compliance

Innovation

• Links shadow library metadata with Goodreads data
• Prioritizes EPUB files for machine-readable quality
• Includes multilingual datasets and evaluates linkage accuracy