Problem
Research questions and friction points this paper is trying to address.
herbarium labels
metadata
digitization
biodiversity informatics
Automated pipeline for herbarium label digitization
🤖 AI Summary
为解决标本标签元数据难以大规模访问的问题，本文提出HERBIOME，一种自动化处理流程，利用YOLOv8、CRAFT Hezar、TrOCR和GPT-4o Mini等技术实现标签信息的自动提取与结构化。
📝 Abstract
Digitized herbarium collections, now comprising over 100 million freely accessible specimen images, have become a critical resource for addressing fundamental questions in ecology and evolutionary biology. Yet the rich metadata encoded in herbarium labels (collector identities, geographic localities, collection dates, and ecological observations) remains largely inaccessible at scale, constraining both biodiversity informatics and the construction of specimen-specific image-text corpora for multimodal AI. We present HERBIOME, a modular end-to-end pipeline for automated herbarium label digitization, integrating YOLOv8-based component detection, CRAFT Hezar word-level text localization, fine-tuned TrOCR for recognition of mixed handwritten and printed text, and GPT-4o Mini for semantic metadata structuring into standardized fields. TrOCR was trained on a multi-source dataset combining general transcription corpora (CREMMA-AN, PictoCatalogs) with herbarium-specific data (RéColNat), achieving a Character Error Rate of 4.05-4.10%. End-to-end evaluation on 450 French herbarium specimens, using a dual-metric framework of Maximum Window Similarity (MWS: 0.614-0.618) and Semantic Metadata Accuracy (SMA: 0.440-0.445), reveals that hybrid training strategies improve semantic fidelity while random sampling maximizes surface similarity, with taxonomic fields remaining the principal bottleneck. By automating the extraction of structured metadata from complex, heterogeneous labels, HERBIOME reduces transcription burden, enables the construction of paired image-text datasets that faithfully capture specimen individuality, which is a prerequisite for next-generation multimodal biodiversity AI systems.
Innovation
Methods, ideas, or system contributions that make the work stand out.
automated herbarium label digitization
end-to-end pipeline
mixed handwritten and printed text recognition
semantic metadata structuring
multimodal AI
🔎 Similar Papers
💼 Related Jobs
No related jobs found.
