OpenSanctions Pairs: Large-Scale Entity Matching with LLMs

2026-02-24

arXiv.org

AI Summary

本文通过发布大规模公开基准OpenSanctions Pairs，使用规则匹配和大语言模型在制裁与开源情报数据上解决实体匹配问题，达到接近实用上限的性能。

Abstract

We release OpenSanctions Pairs, a large-scale entity matching benchmark derived from real-world international sanctions aggregation and analyst deduplication. The dataset contains 755,540 labeled pairs spanning 293 heterogeneous sources across 31 countries, with multilingual and cross-script names, noisy and missing attributes, and set-valued fields typical of compliance workflows. We benchmark a production rule-based matcher (nomenklatura RegressionV1 algorithm) against open- and closed-source LLMs in zero- and few-shot settings. Off-the-shelf LLMs substantially outperform the production rule-based baseline (91.33\% F1), reaching up to 98.95\% F1 (GPT-4o) and 98.23\% F1 with a locally deployable open model (DeepSeek-R1-Distill-Qwen-14B). DSPy MIPROv2 prompt optimization yields consistent but modest gains, while adding in-context examples provides little additional benefit and can degrade performance. Error analysis shows complementary failure modes: the rule-based system over-matches (high false positives), whereas LLMs primarily fail on cross-script transliteration and minor identifier/date inconsistencies. These results indicate that pairwise matching performance is approaching a practical ceiling in this setting, and motivate shifting effort toward pipeline components such as blocking, clustering, and uncertainty-aware review. Code available at https://github.com/chansmi/OSINT_entity_resolution

Problem

• entity matching
• sanctions
• OSINT data
• multilingual
• data diversity

Innovation

• Large-Scale Entity Matching
• LLMs
• OpenSanctions Pairs
• Cross-Script Transliteration
• MIPROv2 Prompt Optimization