Search papers, labs, and topics across Lattice.
OmniTabBench, a new benchmark of 3030 tabular datasets categorized by industry, was introduced to evaluate GBDTs, NNs, and foundation models at scale. Large-scale empirical evaluation on OmniTabBench reveals that no single model family consistently outperforms others across all tabular tasks. Decoupled metafeature analysis identifies dataset properties like size and feature skewness that correlate with the relative performance of different model categories, offering more granular guidance than aggregated metrics.
The myth of a universally superior model for tabular data is busted by a massive 3030-dataset benchmark, revealing nuanced performance dependencies on dataset characteristics.
While traditional tree-based ensemble methods have long dominated tabular tasks, deep neural networks and emerging foundation models have challenged this primacy, yet no consensus exists on a universally superior paradigm. Existing benchmarks typically contain fewer than 100 datasets, raising concerns about evaluation sufficiency and potential selection biases. To address these limitations, we introduce OmniTabBench, the largest tabular benchmark to date, comprising 3030 datasets spanning diverse tasks that are comprehensively collected from diverse sources and categorized by industry using large language models. We conduct an unprecedented large-scale empirical evaluation of state-of-the-art models from all model families on OmniTabBench, confirming the absence of a dominant winner. Furthermore, through a decoupled metafeature analysis, which examines individual properties such as dataset size, feature types, feature and target skewness/kurtosis, we elucidate conditions favoring specific model categories, providing clearer, more actionable guidance than prior compound-metric studies.
