众包、抓取还是生成？构建面向东南亚的多文化视觉语言数据集 SEA-VL
计算机视觉与模式识别
2025-03-19 v2 人工智能
计算与语言
摘要
东南亚（SEA）是语言和文化多样性极其丰富的地区，然而在视觉语言（VL）研究中仍存在显著欠represent。这往往导致人工智能模型无法捕捉 SEA 文化细微差别。为填补这一差距，我们present了 SEA-VL，一个致力于为 SEA 语言开发高质量、文化相关数据的开源计划。通过动员来自 SEA 各国的贡献者，SEA-VL旨在确保更好的文化相关性和多样性，促进 underrepresented 语言在 VL 研究中的包容性。除众包之外，我们的计划进一步探索通过抓取和图像生成来收集文化相关图像的自动化方法。首先，我们发现图像抓取在实现约 ~85% 文化相关性方面，而且比众包更具成本和时间效率。其次，尽管生成式视觉模型取得了显著进展，但合成图像在准确反映 SEA 文化方面仍不可靠。生成的图像往往无法反映该地区的细微传统和文化背景。总体而言，我们收集了 128 万张 SEA 文化相关图像，超过其他现有数据集的 50 倍。通过 SEA-VL，我们旨在弥合 SEA 的 represent 差距，促进开发能够真实代表 SEA 各类文化的包容性 AI 系统。
引用
@article{arxiv.2503.07920,
title = {Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia},
author = {Samuel Cahyawijaya and Holy Lovenia and Joel Ruben Antony Moniz and Tack Hwa Wong and Mohammad Rifqi Farhansyah and Thant Thiri Maung and Frederikus Hudi and David Anugraha and Muhammad Ravi Shulthan Habibi and Muhammad Reza Qorib and Amit Agarwal and Joseph Marvin Imperial and Hitesh Laxmichand Patel and Vicky Feliren and Bahrul Ilmi Nasution and Manuel Antonio Rufino and Genta Indra Winata and Rian Adam Rajagede and Carlos Rafael Catalan and Mohamed Fazli Imam and Priyaranjan Pattnayak and Salsabila Zahirah Pranida and Kevin Pratama and Yeshil Bangera and Adisai Na-Thalang and Patricia Nicole Monderin and Yueqi Song and Christian Simon and Lynnette Hui Xian Ng and Richardy Lobo' Sapan and Taki Hasan Rafi and Bin Wang and Supryadi and Kanyakorn Veerakanjana and Piyalitt Ittichaiwong and Matthew Theodore Roque and Karissa Vincentio and Takdanai Kreangphet and Phakphum Artkaew and Kadek Hendrawan Palgunadi and Yanzhi Yu and Rochana Prih Hastuti and William Nixon and Mithil Bangera and Adrian Xuan Wei Lim and Aye Hninn Khine and Hanif Muhammad Zhafran and Teddy Ferdinan and Audra Aurora Izzani and Ayushman Singh and Evan and Jauza Akbar Krito and Michael Anugraha and Fenal Ashokbhai Ilasariya and Haochen Li and John Amadeo Daniswara and Filbert Aurelian Tjiaranata and Eryawan Presma Yulianrifat and Can Udomcharoenchaikit and Fadil Risdian Ansori and Mahardika Krisna Ihsani and Giang Nguyen and Anab Maulana Barik and Dan John Velasco and Rifo Ahmad Genadi and Saptarshi Saha and Chengwei Wei and Isaiah Flores and Kenneth Ko Han Chen and Anjela Gail Santos and Wan Shen Lim and Kaung Si Phyo and Tim Santos and Meisyarah Dwiastuti and Jiayun Luo and Jan Christian Blaise Cruz and Ming Shan Hee and Ikhlasul Akmal Hanif and M. Alif Al Hakim and Muhammad Rizky Sya'ban and Kun Kerdthaisong and Lester James V. Miranda and Fajri Koto and Tirana Noor Fatyanosa and Alham Fikri Aji and Jostin Jerico Rosal and Jun Kevin and Robert Wijaya and Onno P. Kampman and Ruochen Zhang and Börje F. Karlsson and Peerat Limkonchotiwat},
journal= {arXiv preprint arXiv:2503.07920},
year = {2025}
}
备注
[SEA-VL Dataset] https://huggingface.co/collections/SEACrowd/sea-vl-multicultural-vl-dataset-for-southeast-asia-67cf223d0c341d4ba2b236e7 [Appendix J] https://github.com/SEACrowd/seacrowd.github.io/blob/master/docs/SEA_VL_Appendix_J.pdf