Search papers, labs, and topics across Lattice.
The paper introduces SQLFlex, a query rewriting framework that combines grammar-based parsing with LLM-based segmentation to achieve robust parsing across diverse SQL dialects. SQLFlex decomposes hierarchical parsing into sequential segmentation tasks at clause and expression levels, leveraging LLMs for segmentation and grammar-based methods for validation. Experiments demonstrate that SQLFlex outperforms existing tools like SQLFluff and SQLess in SQL linting and test-case reduction, achieving high parsing accuracy (91.55% - 100%) across eight SQL dialects.
LLMs can parse almost any SQL dialect by segmenting queries into clauses and expressions, then validating with grammar-based methods.
SQL is a widely adopted language for querying data, which has led to the development of various SQL analysis and rewriting tools. However, due to the diversity of SQL dialects, such tools often fail when encountering unrecognized dialect-specific syntax. While Large Language Models (LLMs) have shown promise in understanding SQL queries, their inherent limitations in handling hierarchical structures and hallucination risks limit their direct applicability in parsing. To address these limitations, we propose SQLFlex, a novel query rewriting framework that integrates grammar-based parsing with LLM-based segmentation to parse diverse SQL dialects robustly. Our core idea is to decompose hierarchical parsing to sequential segmentation tasks, which better aligns with the strength of LLMs and improves output reliability through validation checks. Specifically, SQLFlex uses clause-level segmentation and expression-level segmentation as two strategies that decompose elements on different levels of a query. We extensively evaluated SQLFlex on both real-world use cases and in a standalone evaluation. In SQL linting, SQLFlex outperforms SQLFluff in ANSI mode by 63.68% in F1 score while matching its dialect-specific mode performance. In test-case reduction, SQLFlex outperforms SQLess by up to 10 times in simplification rate. In the standalone evaluation, it parses 91.55% to 100% of queries across eight distinct dialects, outperforming all baseline parsers. We believe SQLFlex can serve as a foundation for many query analysis and rewriting use cases.
