Autoregressive large language models (LLMs) scale well by expressing diverse tasks as sequences of discrete natural-language tokens and training with next-token prediction, which unifies comprehension and generation under self-supervision. Extending this paradigm to multimodal data requires a shared, discrete representation across modalities. However, most vision-language models (VLMs) still rely on a hybrid interface: discrete text tokens paired with continuous Vision Transformer (ViT) features. Because supervision is largely text-driven, these models are often biased toward understanding and cannot fully leverage large-scale self-supervised learning on non-text data. Recent work has explored discrete visual tokenization to enable fully autoregressive multimodal modeling, showing promising progress toward unified understanding and generation. Yet existing discrete vision tokens frequently lose information due to limited code capacity, resulting in noticeably weaker understanding than continuous-feature VLMs. We present Kelix, a fully discrete autoregressive unified model that closes the understanding gap between discrete and continuous visual representations.
@article{arxiv.2602.09843,
title = {Kelix Technical Report},
author = {Boyang Ding and Chenglong Chu and Dunju Zang and Han Li and Jiangxia Cao and Kun Gai and Muhao Wei and Ruiming Tang and Shiyao Wang and Siyang Mao and Xinchen Luo and Yahui Liu and Zhixin Ling and Zhuoran Yang and Ziming Li and Chengru Song and Guorui Zhou and Guowang Zhang and Hao Peng and Hao Wang and Jiaxin Deng and Jin Ouyang and Jinghao Zhang and Lejian Ren and Qianqian Wang and Qigen Hu and Tao Wang and Xingmei Wang and Yiping Yang and Zixing Zhang and Ziqi Wang},
journal= {arXiv preprint arXiv:2602.09843},
year = {2026}
}