Search papers, labs, and topics across Lattice.
This paper presents a text-to-SQL system leveraging large language models (LLMs) to facilitate natural language queries on the ALeRCE astronomical database. Utilizing a structured framework that includes schema linking, query classification, prompt decomposition, and self-correction, the system significantly improves SQL query generation accuracy compared to a direct-inference baseline. Notably, the best-performing model, Claude Opus 4.6, achieves a perfect-match rate of 0.97 for row identifiers on simple queries, demonstrating the potential of LLMs in complex data querying tasks.
A step-by-step text-to-SQL framework enables LLMs to achieve near-perfect accuracy on simple queries, revealing the untapped potential of natural language interfaces in astronomical data exploration.
We develop a text-to-SQL (structured query language) system based on large language models (LLMs) using in-context learning and apply it to the Automatic Learning for the Rapid Classification of Events (ALeRCE) astronomical database. ALeRCE is a community broker for the Zwicky Transient Facility and the Vera C. Rubin Observatory. The system enables users to query the database in natural language (NL) and generates executable SQL queries. To develop and evaluate the system, we constructed a dataset of 110 NL/SQL pairs. We propose a step-by-step generation framework comprising four modules: schema linking, query classification, prompt decomposition, and self-correction. The performance of thirteen LLMs is evaluated using in-context learning and prompt engineering techniques. Text-to-SQL performance is assessed using the perfect-match (PM) rate for row identifiers (e.g., object identifiers) and column identifiers (i.e., column names). The proposed step-by-step framework consistently outperforms a direct-inference baseline, while the self-correction module consistently reduces execution errors. For Claude Opus 4.6, PM performance on row (column) identifiers is high for simple queries, reaching 0.97 (0.94), and decreases with query complexity to 0.44 (0.72) for medium queries and 0.59 (0.49) for hard queries. Among the thirteen evaluated models, the best-performing LLMs for the text-to-SQL task are Claude Opus 4.6, Gemini 2.5 Pro, Gemini 3 Flash, and GPT-5.2-Codex.
