Search papers, labs, and topics across Lattice.
This paper introduces the ProgramTab framework, which enhances table reasoning capabilities of large language models (LLMs) by integrating programmatic paradigms for preprocessing tabular data. By leveraging in-context learning and Python code for data extraction and SQL generation, ProgramTab addresses the challenges posed by large, unstructured web tables that hinder traditional text-to-SQL approaches. Experimental results show that ProgramTab significantly outperforms existing LLM-based methods on various table reasoning datasets, highlighting its effectiveness in complex reasoning tasks.
ProgramTab outperforms all LLM-based baselines in table reasoning by transforming unstructured data into actionable insights through programmatic preprocessing.
Table-based reasoning with large language models (LLMs), which requires reasoning based on natural language questions and structured tabular data, has gained widespread attention. However, a series of issues still constrain the application of this task. The previous approaches suffered from significant performance degradation when faced with large tables due to the difficulty of long text modeling and the limitation of input length for LLMs. The text-to-SQL approach is used to efficiently extract key information from tables and generate smaller sub-tables. However, tabular data, especially web tables, often lack the necessary structure and consistency, making them unsuitable for performing mathematical logic operations using SQL queries. We propose the ProgramTab framework, which guides LLMs employing in-context learning to perform tabular data preprocessing with Python code, as well as the momentous contents extraction with row and column extraction and SQL generation. The experiment results on table reasoning datasets demonstrate that the ProgramTab framework effectively deals with table-based reasoning tasks and outperforms all LLM-based baselines.
