PaLI-X：关于多语言视觉与语言模型的扩展
计算机视觉与模式识别
2023-05-31 v1 计算与语言
机器学习
摘要
我们给出了多语言视觉与语言模型 PaLI-X 在组件规模及其训练任务混合广度两方面进行扩展的训练方案与结果。我们的模型在广泛多样且复杂的任务上达到了新的性能水平，包括多个基于图像的 captioning 与问答任务、基于图像的文档理解以及少样本（上下文内）学习，还有目标检测、视频问答和视频 captioning。PaLI-X 在绝大多数所考虑的视觉与语言基准（其中 25+ 个）上推进了当前最优水平（SOTA）。最后，我们观察到涌现能力，例如复杂计数与多语言目标检测，这些任务并未显式包含在训练混合中。
引用
@article{arxiv.2305.18565,
title = {PaLI-X: On Scaling up a Multilingual Vision and Language Model},
author = {Xi Chen and Josip Djolonga and Piotr Padlewski and Basil Mustafa and Soravit Changpinyo and Jialin Wu and Carlos Riquelme Ruiz and Sebastian Goodman and Xiao Wang and Yi Tay and Siamak Shakeri and Mostafa Dehghani and Daniel Salz and Mario Lucic and Michael Tschannen and Arsha Nagrani and Hexiang Hu and Mandar Joshi and Bo Pang and Ceslee Montgomery and Paulina Pietrzyk and Marvin Ritter and AJ Piergiovanni and Matthias Minderer and Filip Pavetic and Austin Waters and Gang Li and Ibrahim Alabdulmohsin and Lucas Beyer and Julien Amelot and Kenton Lee and Andreas Peter Steiner and Yang Li and Daniel Keysers and Anurag Arnab and Yuanzhong Xu and Keran Rong and Alexander Kolesnikov and Mojtaba Seyedhosseini and Anelia Angelova and Xiaohua Zhai and Neil Houlsby and Radu Soricut},
journal= {arXiv preprint arXiv:2305.18565},
year = {2023}
}