Multilingual corpora for the study of new concepts in the social sciences and humanities:

2025-12-08

AI Summary

This study addresses the scarcity of multilingual corpora supporting emerging concepts—such as “non-technological innovation”—in the humanities and social sciences (HSS). To tackle this, we propose a hybrid multilingual corpus construction methodology that integrates corporate websites and annual reports, combining automatic language identification, domain-adapted content filtering, relevant paragraph extraction, expert-lexicon-driven contextual block identification, thematic annotation, and enriched structured metadata. Our key contribution is the first systematic construction of a high-quality, multilingual corpus specifically designed for HSS emerging concepts, accompanied by a parallel English supervised dataset with fine-grained thematic labels. The resulting resource enables cross-lingual lexical variation analysis, training of multilingual NLP models, and empirical social science research. It is both reusable and extensible, effectively bridging a critical gap between domain-specific knowledge modeling and computational linguistics applications.

Technology Category

• Application Domains: Humanities & Computational Social Science
• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

This article presents a hybrid methodology for building a multilingual corpus designed to support the study of emerging concepts in the humanities and social sciences (HSS), illustrated here through the case of ``non-technological innovation''. The corpus relies on two complementary sources: (1) textual content automatically extracted from company websites, cleaned for French and English, and (2) annual reports collected and automatically filtered according to documentary criteria (year, format, duplication). The processing pipeline includes automatic language detection, filtering of non-relevant content, extraction of relevant segments, and enrichment with structural metadata. From this initial corpus, a derived dataset in English is created for machine learning purposes. For each occurrence of a term from the expert lexicon, a contextual block of five sentences is extracted (two preceding and two following the sentence containing the term). Each occurrence is annotated with the thematic category associated with the term, enabling the construction of data suitable for supervised classification tasks. This approach results in a reproducible and extensible resource, suitable both for analyzing lexical variability around emerging concepts and for generating datasets dedicated to natural language processing applications.

Problem

• Build a multilingual corpus for studying emerging HSS concepts
• Create a machine-learning dataset from expert lexicon occurrences
• Enable lexical analysis and NLP applications via reproducible resources

Innovation

• Hybrid methodology combining website texts and reports
• Processing pipeline with language detection and filtering
• Contextual extraction for supervised classification datasets