MegaHan97K: A large-scale dataset for mega-category Chinese character recognition with over 97K categories

2025-05-01

Pattern Recognition

AI Summary

Existing Chinese character recognition benchmarks suffer from limited class scale, incomplete coverage, and scarce annotations. To address these issues, this work introduces the first large-scale, fine-grained benchmark comprising 97,342 Chinese characters—including rare, variant, and archaic forms—exceeding 97K classes. We propose a multi-source heterogeneous acquisition and semantic consistency verification framework, integrating Unicode Extension Zones, ancient-text OCR outputs, synthetic handwritten samples, and expert manual validation, augmented with structured semantic labels and stroke-order encoding. The resulting dataset achieves 99.8% annotation accuracy. Empirical evaluation demonstrates substantial performance gains for state-of-the-art models on long-tail classes, notably improving robustness and generalization. This benchmark establishes critical infrastructure for high-accuracy Chinese character recognition, directly supporting cultural heritage preservation and digital humanities research.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Large Vision Models
• Knowledge Representation and Reasoning: Knowledge Acquisition

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Representation, semantic annotation, enhancement, enrichments, access and/or integration of a variety of data on the Web
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Problem

• Addressing mega-category Chinese character recognition challenges
• Providing a comprehensive dataset with 97,455 character categories
• Solving long-tail distribution and zero-shot learning difficulties

Innovation

• Largest dataset for Chinese character recognition
• Supports GB18030-2022 with 97,455 categories
• Addresses long-tail distribution with balanced subsets