🤖 AI Summary
This work addresses the high barrier posed by SQL-based querying of astronomical databases for non-expert users. The authors propose a large language model (LLM)-driven text-to-SQL system enabling natural language queries over the ALeRCE database. Their approach employs a four-module, stepwise generation framework—comprising schema linking, query classification, prompt decomposition, and self-correction—which substantially outperforms end-to-end baselines and significantly reduces execution errors. Evaluation on 110 annotated samples across 13 LLMs demonstrates strong performance: on models such as Claude Opus 4.6, exact match rates for row and column identifiers reach 0.97 and 0.94, respectively, for simple queries, while maintaining robustness on complex queries.
📝 Abstract
We develop a text-to-SQL (structured query language) system based on large language models (LLMs) using in-context learning and apply it to the Automatic Learning for the Rapid Classification of Events (ALeRCE) astronomical database. ALeRCE is a community broker for the Zwicky Transient Facility and the Vera C. Rubin Observatory. The system enables users to query the database in natural language (NL) and generates executable SQL queries. To develop and evaluate the system, we constructed a dataset of 110 NL/SQL pairs. We propose a step-by-step generation framework comprising four modules: schema linking, query classification, prompt decomposition, and self-correction. The performance of thirteen LLMs is evaluated using in-context learning and prompt engineering techniques. Text-to-SQL performance is assessed using the perfect-match (PM) rate for row identifiers (e.g., object identifiers) and column identifiers (i.e., column names). The proposed step-by-step framework consistently outperforms a direct-inference baseline, while the self-correction module consistently reduces execution errors. For Claude Opus 4.6, PM performance on row (column) identifiers is high for simple queries, reaching 0.97 (0.94), and decreases with query complexity to 0.44 (0.72) for medium queries and 0.59 (0.49) for hard queries. Among the thirteen evaluated models, the best-performing LLMs for the text-to-SQL task are Claude Opus 4.6, Gemini 2.5 Pro, Gemini 3 Flash, and GPT-5.2-Codex.