Jais与Jais-chat：以阿拉伯语为中心的基础与指令微调开放生成式大语言模型
计算与语言
2023-10-02 v2 人工智能
机器学习
摘要
我们介绍Jais与Jais-chat，新的最先进的以阿拉伯语为中心的基础与指令微调开放生成式大语言模型（LLM）。这些模型基于GPT-3仅解码器架构，并在阿拉伯语与英语文本混合（包括多种编程语言的源代码）上预训练。拥有130亿参数，基于广泛评估，它们在阿拉伯语上的知识与推理能力显著优于任何现有开放阿拉伯语及多语言模型。此外，尽管在英语上训练数据少得多，这些模型在与同等规模的以英语为中心的开放模型相比时仍具竞争力。我们详细描述了模型的训练、微调、安全对齐与评估。我们发布两个开放版本——基础Jais模型与指令微调的Jais-chat变体——旨在促进阿拉伯语LLM的研究。可用地址 https://huggingface.co/inception-mbzuai/jais-13b-chat
引用
@article{arxiv.2308.16149,
title = {Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models},
author = {Neha Sengupta and Sunil Kumar Sahu and Bokang Jia and Satheesh Katipomu and Haonan Li and Fajri Koto and William Marshall and Gurpreet Gosal and Cynthia Liu and Zhiming Chen and Osama Mohammed Afzal and Samta Kamboj and Onkar Pandit and Rahul Pal and Lalit Pradhan and Zain Muhammad Mujahid and Massa Baali and Xudong Han and Sondos Mahmoud Bsharat and Alham Fikri Aji and Zhiqiang Shen and Zhengzhong Liu and Natalia Vassilieva and Joel Hestness and Andy Hock and Andrew Feldman and Jonathan Lee and Andrew Jackson and Hector Xuguang Ren and Preslav Nakov and Timothy Baldwin and Eric Xing},
journal= {arXiv preprint arXiv:2308.16149},
year = {2023}
}
备注
Arabic-centric, foundation model, large-language model, LLM, generative model, instruction-tuned, Jais, Jais-chat