Leveraging LLMs and attention-mechanism for automatic annotation of historical maps

2025-04-15

AI Summary

To address the lack of structured annotations and poor scalability for machine interpretation in historical map digitization, this paper proposes an LLM-guided, attention-mechanism-enhanced framework for automatic semantic annotation. Departing from pixel-level ground-truth supervision, our method leverages large language models to generate coarse-grained semantic labels and employs attention-driven multi-scale image patch feature alignment to refine these labels, integrating multi-resolution processing and knowledge distillation. On the Wood and Settlement categories, the framework achieves IoU scores of 84.2% and 72.0%, respectively, with recall exceeding 90% and precision reaching 87.1% and 79.5%. To the best of our knowledge, this is the first end-to-end semantic parsing approach for historical maps that operates without fine-grained manual annotations. The method significantly improves robustness and cross-domain generalization capability, enabling scalable, annotation-efficient interpretation of heterogeneous historical cartographic materials.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Computer Vision: Segmentation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Historical maps are essential resources that provide insights into the geographical landscapes of the past. They serve as valuable tools for researchers across disciplines such as history, geography, and urban studies, facilitating the reconstruction of historical environments and the analysis of spatial transformations over time. However, when constrained to analogue or scanned formats, their interpretation is limited to humans and therefore not scalable. Recent advancements in machine learning, particularly in computer vision and large language models (LLMs), have opened new avenues for automating the recognition and classification of features and objects in historical maps. In this paper, we propose a novel distillation method that leverages LLMs and attention mechanisms for the automatic annotation of historical maps. LLMs are employed to generate coarse classification labels for low-resolution historical image patches, while attention mechanisms are utilized to refine these labels to higher resolutions. Experimental results demonstrate that the refined labels achieve a high recall of more than 90%. Additionally, the intersection over union (IoU) scores--84.2% for Wood and 72.0% for Settlement--along with precision scores of 87.1% and 79.5%, respectively, indicate that most labels are well-aligned with ground-truth annotations. Notably, these results were achieved without the use of fine-grained manual labels during training, underscoring the potential of our approach for efficient and scalable historical map analysis.

Problem

• Automating annotation of historical maps using LLMs
• Enhancing label resolution via attention mechanisms
• Achieving high accuracy without manual fine-grained labels

Innovation

• Leveraging LLMs for coarse historical map annotation
• Using attention mechanisms to refine annotation resolution
• Achieving high accuracy without fine-grained manual labels