Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To resolve the tradeoff between semantic-blind tabular foundation models and unstable from-scratch AutoML agents, the authors pair a pretrained tabular model with an LLM agent that iteratively evolves only the surrounding data engineering pipeline. By constraining the agent's search space to data cleaning, feature engineering, context selection, and post-processing guided by metadata and validation feedback, the system avoids the noisy overfitting typical of joint architecture-hyperparameter search. TabFM-Auto secured the top five positions on the 51-dataset TabArena benchmark (+228 Elo gain), outperformed all competing agents on MLE-Bench's tabular tracks, and produced pipelines that transferred zero-shot to other frozen foundation models.
Frozen tabular foundation models can dominate competitive ML benchmarks without parameter updates if an LLM agent is tasked strictly with evolving the semantic data pipeline around them.
Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, task descriptions, and auxiliary files that carry dataset semantics. Meanwhile, self-evolving machine learning engineering (MLE) agents train models from scratch on each dataset, yet jointly searching over features, architectures, and hyperparameters is noisy and prone to overfitting. We introduce TabFM-Auto, which pairs a tabular foundation model, TabFM, with a language model agent that evolves the data pipeline around it. Guided by dataset metadata and validation feedback, TabFM-Auto iteratively refines data cleaning, feature engineering, context selection, and post-processing to reduce TabFM's error. Across all 51 datasets of the TabArena benchmark, five TabFM-Auto configurations with different agents and language models take the top five overall positions, and the best raises TabFM from 1785 to 2013 Elo. The discovered pipelines also transfer to other frozen tabular foundation models (+69 to +143 Elo) with no further search. On the 8 tabular competitions of MLE-Bench, TabFM-Auto ranks first overall among MLE agents.