Search papers, labs, and topics across Lattice.
This study critically evaluates the consistency of Tabular Foundation Models (TFMs) in their predictions by introducing two key requirements: marginalization consistency and factorization consistency. The authors find that all evaluated TFMs fail to meet these criteria for both classification and regression tasks across various datasets, highlighting a significant gap in the reliability of these models. This investigation underscores the limitations of TFMs in accurately representing joint distributions, which is crucial for their application in real-world scenarios.
TFMs, despite being the leading approach for tabular predictions, fail to consistently represent joint distributions, raising concerns about their reliability in practical applications.
Tabular Foundation Models (TFMs) are currently the best approach to tabular prediction problems. They are constructed as transformers that approximate the Bayesian posterior predictive distribution based on a pre-training prior. These univariate predictors can be converted into multivariate ones autoregressively by sampling one target and adding it to the features. However, the faithfulness of the resulting joint has not been investigated. Furthermore, TFMs cannot be evaluated against the posterior itself, at least not on real-world datasets, because the ground-truth distribution is unknown. We therefore propose asking a different question: could a model's predictions result from any joint distribution? To answer this question, we pose two requirements that any such model must satisfy. The first is marginalization consistency, which demands that marginalized conditionals are equal to directly predicted marginals. The second is factorization consistency, which demands that different factorization orders result in equal joint distributions. Every TFM that we evaluate violates both of these requirements for both classification and regression across all datasets.
