Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models
Abstract
Since the seminal work of TabPFN, research on tabular foundation models (TFMs) based on in-context learning (ICL) has challenged long-standing paradigms in machine learning. Without seeing any real-world data, models pretrained on purely synthetic datasets generalize remarkably well across diverse datasets, often using only a moderate number of in-context examples. This shifts the focus in tabular machine learning from model architecture design to the design of synthetic datasets, or, more precisely, to the prior distributions that generate them. Yet the guiding principles for prior design remain poorly understood. This work marks the first attempt to address the gap. We systematically investigate and identify key properties of synthetic priors that allow pretrained TFMs to generalize well. Based on these insights, we introduce Mitra, a TFM trained on a curated mixture of synthetic priors selected for their diversity, distinctiveness, and performance on real-world tabular data. Mitra consistently outperforms state-of-the-art TFMs, such as TabPFNv2 and TabICL, across both classification and regression benchmarks, with better sample efficiency.
Cite
@article{arxiv.2510.21204,
title = {Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models},
author = {Xiyuan Zhang and Danielle C. Maddix and Junming Yin and Nick Erickson and Abdul Fatir Ansari and Boran Han and Shuai Zhang and Leman Akoglu and Christos Faloutsos and Michael W. Mahoney and Cuixiong Hu and Huzefa Rangwala and George Karypis and Bernie Wang},
journal= {arXiv preprint arXiv:2510.21204},
year = {2025}
}
Comments
NeurIPS 2025. We released both classifier (autogluon/mitra-classifier) and regressor (autogluon/mitra-regressor) model weights on HuggingFace