Automating SPARQL Query Translations between DBpedia and Wikidata

2025-07-14

AI Summary

This work addresses the lack of systematic evaluation in SPARQL cross-schema query translation for knowledge graph (KG) interoperability. We introduce the first SPARQL-to-SPARQL translation benchmark targeting heterogeneous KGs—specifically, DBpedia/Wikidata and DBLP/OpenAlex. Leveraging large language models (LLMs) including Llama-3-8B, DeepSeek-R1-Distill-Llama-70B, and Mistral-Large-Instruct-2407, we propose a multi-prompting strategy (zero-shot, few-shot, and chain-of-thought) to assess translation performance under asymmetric schema mappings. Empirical results reveal a significant performance gap: translation from Wikidata to DBpedia achieves markedly higher accuracy than the reverse direction. Our key contributions are threefold: (1) the first dedicated benchmark for cross-KG SPARQL translation; (2) an optimized translation pipeline tailored to asymmetric schema mappings; and (3) scalability validation across encyclopedic (Wikidata/DBpedia) and scholarly (DBLP/OpenAlex) KG domains.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Description Logics
• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

This paper investigates whether state-of-the-art Large Language Models (LLMs) can automatically translate SPARQL between popular Knowledge Graph (KG) schemas. We focus on translations between the DBpedia and Wikidata KG, and later on DBLP and OpenAlex KG. This study addresses a notable gap in KG interoperability research by rigorously evaluating LLM performance on SPARQL-to-SPARQL translation. Two benchmarks are assembled, where the first align 100 DBpedia-Wikidata queries from QALD-9-Plus; the second contains 100 DBLP queries aligned to OpenAlex, testing generalizability beyond encyclopaedic KGs. Three open LLMs: Llama-3-8B, DeepSeek-R1-Distill-Llama-70B, and Mistral-Large-Instruct-2407 are selected based on their sizes and architectures and tested with zero-shot, few-shot, and two chain-of-thought variants. Outputs were compared with gold answers, and resulting errors were categorized. We find that the performance varies markedly across models and prompting strategies, and that translations for Wikidata to DBpedia work far better than translations for DBpedia to Wikidata.

Problem

• Automating SPARQL query translations between DBpedia and Wikidata
• Evaluating LLM performance on SPARQL-to-SPARQL translation
• Assessing generalizability across different Knowledge Graph schemas

Innovation

• Uses LLMs for SPARQL-to-SPARQL translation
• Evaluates three open LLMs with varied strategies
• Benchmarks DBpedia-Wikidata and DBLP-OpenAlex queries