Keywords are not always the key: A metadata field analysis for natural language search on open data portals

2025-09-17

AI Summary

Keyword search in open data portals suffers from incomplete and inconsistent metadata, as well as a semantic gap between user queries and domain-specific terminology, leading to poor natural language query performance. This work focuses on the critical role of metadata fields—particularly descriptive text—in enabling semantic alignment and proposes a large language model (LLM)-based method to automatically generate high-quality dataset descriptions that bridge the gap between user intent and structured metadata. Through controlled ablation studies and simulated natural language queries, we systematically evaluate retrieval performance across varying metadata configurations and prompting strategies. Results demonstrate that LLM-generated descriptions significantly improve both precision and recall, validating the effectiveness and practicality of generative approaches for enhancing open data discoverability. Our approach establishes a reproducible, scalable paradigm for metadata enrichment in open data ecosystems.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Open data portals are essential for providing public access to open datasets. However, their search interfaces typically rely on keyword-based mechanisms and a narrow set of metadata fields. This design makes it difficult for users to find datasets using natural language queries. The problem is worsened by metadata that is often incomplete or inconsistent, especially when users lack familiarity with domain-specific terminology. In this paper, we examine how individual metadata fields affect the success of conversational dataset retrieval and whether LLMs can help bridge the gap between natural queries and structured metadata. We conduct a controlled ablation study using simulated natural language queries over real-world datasets to evaluate retrieval performance under various metadata configurations. We also compare existing content of the metadata field 'description' with LLM-generated content, exploring how different prompting strategies influence quality and impact on search outcomes. Our findings suggest that dataset descriptions play a central role in aligning with user intent, and that LLM-generated descriptions can support effective retrieval. These results highlight both the limitations of current metadata practices and the potential of generative models to improve dataset discoverability in open data portals.

Problem

• Analyzing metadata fields for natural language search on open data portals
• Evaluating how LLMs bridge natural queries and structured metadata
• Assessing impact of incomplete metadata on dataset discoverability

Innovation

• LLM-generated metadata enhances dataset discoverability
• Ablation study evaluates metadata field effectiveness
• Conversational queries bridge natural language and structured data