Search papers, labs, and topics across Lattice.
Xiaomi-TabLDM is a novel tabular large data foundation model designed for classification and regression that excels in prediction accuracy without the need for task-specific fine-tuning. By leveraging synthetic data generated from structural causal models and employing a three-stage training strategy with advanced techniques like dual-stream feature grouping and sparse Mixture-of-Experts, the model achieves superior performance across multiple benchmarks. Notably, it ranks first on OpenML-CTR23 and demonstrates a significant efficiency advantage, using 82% less training time and 68% less prediction time compared to the leading model in its category.
Achieving top-tier regression performance while drastically reducing computational costs, Xiaomi-TabLDM sets a new benchmark for efficiency in tabular data modeling.
We introduce Xiaomi-TabLDM, a tabular large data foundation model for classification and regression via in-context learning, which delivers superior prediction accuracy without requiring task-specific fine-tuning. Pretrained exclusively on synthetic data generated from structural causal models (SCMs), our model enables more flexible context utilization and more efficient capacity scaling. i) A new performance standard. Strong regression performance across benchmarks: Xiaomi-TabLDM ranks 1st on OpenML-CTR23 and 2nd on regression across TALENT, TabArena, and BCCO, demonstrating consistently strong regression performance across four complementary benchmark suites. Favorable performance--efficiency trade-off: Xiaomi-TabLDM combines strong predictive performance with substantially lower computational cost. For example, on TabArena regression, it achieves the second-highest Elo while using 82% less training time and 68% less prediction time than the top-ranked TabFM. ii) Large-scale synthetic pretraining. Xiaomi-TabLDM expands the coverage and diversity of synthetic tabular data used for pretraining. We also adopt a three-stage training strategy together with dual-stream feature grouping, lightweight Attention Residual, and sparse Mixture-of-Experts, enabling Xiaomi-TabLDM to learn richer feature interactions and expert specialization across diverse tabular tasks. iii) Test-time scaling. Xiaomi-TabLDM further extends tabular prediction through test-time compute scaling, where allocating additional computation at inference time consistently improves predictive performance over the base model.
