Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors evaluate whether LLMs can infer both data domains and downstream prediction task types given only dataset metadata and a user-specified target column, introducing a benchmark of 625 tabular and time-series datasets. Automating this step resolves a persistent human-in-the-loop bottleneck in end-to-end AutoML pipeline generation where static heuristics frequently fail on ambiguous schemas. LLM-based inference outperforms AutoGluon's heuristics on tabular data (0.98 vs. 0.93 macro F1) and maintains 0.90 macro F1 in cross-domain settings, though performance drops to 0.75 when using smaller local models.
Even when provided with only target column metadata, LLMs outperform specialized AutoML heuristics like AutoGluon at inferring machine learning task types (0.98 vs. 0.93 macro F1), unlocking fully autonomous pipeline configuration.
Machine learning task type identification is essential for constructing valid ML pipelines, yet in practice it is typically specified manually. We investigate whether large language models (LLMs) can infer both the data domain and the downstream prediction task directly from dataset-level information when only the target feature is provided by the user. Together with our LLM-based system we also release an annotated benchmark comprising 625 public tabular and time series datasets. We evaluate the proposed approach in three settings: (i) tabular datasets in comparison with established AutoML heuristics, (ii) cross-domain evaluation across tabular and time series datasets, and (iii) a practical deployment scenario using smaller local models. The results show consistent advantages for LLM-based task type identification, with increasing difficulty in heterogeneous and resource-constrained settings. LLM-based approaches outperform AutoGluon in the tabular setting, reaching 0.98 F1 macro compared to 0.93. In the cross-domain setting, the best model achieves 0.90 F1 macro, while smaller locally deployable models reach 0.75, indicating a trade-off between deployment feasibility and accuracy.