TabKANet: Tabular Data Modeling with Kolmogorov-Arnold Network and Transformer
Abstract
Tabular data is the most common type of data in real-life scenarios. In this study, we propose the TabKANet model for tabular data modeling, which targets the bottlenecks in learning from numerical content. We constructed a Kolmogorov-Arnold Network (KAN) based Numerical Embedding Module and unified numerical and categorical features encoding within a Transformer architecture. TabKANet has demonstrated stable and significantly superior performance compared to Neural Networks (NNs) across multiple public datasets in binary classification, multi-class classification, and regression tasks. Its performance is comparable to or surpasses that of Gradient Boosted Decision Tree models (GBDTs). Our code is publicly available on GitHub: https://github.com/AI-thpremed/TabKANet.
Keywords
Cite
@article{arxiv.2409.08806,
title = {TabKANet: Tabular Data Modeling with Kolmogorov-Arnold Network and Transformer},
author = {Weihao Gao and Zheng Gong and Zhuo Deng and Fuju Rong and Chucheng Chen and Lan Ma},
journal= {arXiv preprint arXiv:2409.08806},
year = {2024}
}
Comments
13 pages,5 figures