Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SAFAARI, a multi-agent framework designed to enhance Natural Language to SQL (NL-to-SQL) systems by automating schema linking through specialized agents. By employing a novel metric, SEAL, to evaluate performance, SAFAARI achieves an 81.66% score, demonstrating significant improvements in datapoint accuracy and schema-linking precision. The framework not only reduces development time by 8x but also adapts effectively across various customer support domains, addressing a critical bottleneck in accessing enterprise data.
Automating schema linking in NL-to-SQL systems can cut development time by 8x while boosting accuracy, revolutionizing customer support data access.
The evolution of customer support systems is rapidly advancing with agentic chatbots, yet these systems face significant limitations when accessing enterprise data without predefined API endpoints. This paper presents SAFAARI (Schema-Aware Framework for Accelerated Advertiser Response Intelligence), a multi-agent framework that addresses the critical bottleneck of schema linking in Natural Language to SQL (NL-to-SQL) systems through specialized content, metadata, and orchestration agents. We also introduce SEAL (Schema Evaluation and Accuracy in Language-to-SQL), a novel composite metric that holistically evaluates system performance while penalizing inconsistent results. Through systematic experimentation with five feature set configurations, SAFAARI achieves an 81.66% SEAL score (6.65% improvement over baseline), with notable gains in datapoint accuracy (5.51%) and schema-linking precision (4.69%). The framework's effectiveness is validated through human-in-the-loop evaluation with domain experts, which proves its adaptability across diverse support domains. By automating the labor-intensive process of schema linking and query generation, our framework demonstrates 8x reduction in development time while maintaining high accuracy. The solution streamlines API development and enhances self-service capabilities, particularly benefiting customer support enterprises with complex data ecosystems.