基础模型的机遇与风险
机器学习
2022-07-14 v3 人工智能
计算机与社会
摘要
随着在大规模广泛数据上训练且可适配广泛下游任务的模型（如BERT、DALL-E、GPT-3）的兴起，AI正在经历范式转变。我们称这些模型为基础模型，以强调其关键核心却尚不完备的特性。本报告全面阐述了基础模型的机遇与风险，涵盖其能力（如语言、视觉、机器人、推理、人机交互）和技术原理（如模型架构、训练流程、数据、系统、安全、评估、理论），到其应用（如法律、医疗、教育）和社会影响（如不平等、滥用、经济与环境影响、法律与伦理考量）。尽管基础模型基于标准深度学习和迁移学习，其规模带来了新的涌现能力，其在众多任务上的有效性也促使同质化。同质化提供了强大的杠杆，但要求谨慎，因为基础模型的缺陷会被所有下游适配模型所继承。尽管基础模型即将被广泛部署，我们目前仍缺乏对其工作机制、失败时机乃至因其涌现特性而具备的能力的清晰理解。为应对这些问题，我们认为许多关于基础模型的关键研究将需要与其根本的社会技术性质相称的深层跨学科协作。
引用
@article{arxiv.2108.07258,
title = {On the Opportunities and Risks of Foundation Models},
author = {Rishi Bommasani and Drew A. Hudson and Ehsan Adeli and Russ Altman and Simran Arora and Sydney von Arx and Michael S. Bernstein and Jeannette Bohg and Antoine Bosselut and Emma Brunskill and Erik Brynjolfsson and Shyamal Buch and Dallas Card and Rodrigo Castellon and Niladri Chatterji and Annie Chen and Kathleen Creel and Jared Quincy Davis and Dora Demszky and Chris Donahue and Moussa Doumbouya and Esin Durmus and Stefano Ermon and John Etchemendy and Kawin Ethayarajh and Li Fei-Fei and Chelsea Finn and Trevor Gale and Lauren Gillespie and Karan Goel and Noah Goodman and Shelby Grossman and Neel Guha and Tatsunori Hashimoto and Peter Henderson and John Hewitt and Daniel E. Ho and Jenny Hong and Kyle Hsu and Jing Huang and Thomas Icard and Saahil Jain and Dan Jurafsky and Pratyusha Kalluri and Siddharth Karamcheti and Geoff Keeling and Fereshte Khani and Omar Khattab and Pang Wei Koh and Mark Krass and Ranjay Krishna and Rohith Kuditipudi and Ananya Kumar and Faisal Ladhak and Mina Lee and Tony Lee and Jure Leskovec and Isabelle Levent and Xiang Lisa Li and Xuechen Li and Tengyu Ma and Ali Malik and Christopher D. Manning and Suvir Mirchandani and Eric Mitchell and Zanele Munyikwa and Suraj Nair and Avanika Narayan and Deepak Narayanan and Ben Newman and Allen Nie and Juan Carlos Niebles and Hamed Nilforoshan and Julian Nyarko and Giray Ogut and Laurel Orr and Isabel Papadimitriou and Joon Sung Park and Chris Piech and Eva Portelance and Christopher Potts and Aditi Raghunathan and Rob Reich and Hongyu Ren and Frieda Rong and Yusuf Roohani and Camilo Ruiz and Jack Ryan and Christopher Ré and Dorsa Sadigh and Shiori Sagawa and Keshav Santhanam and Andy Shih and Krishnan Srinivasan and Alex Tamkin and Rohan Taori and Armin W. Thomas and Florian Tramèr and Rose E. Wang and William Wang and Bohan Wu and Jiajun Wu and Yuhuai Wu and Sang Michael Xie and Michihiro Yasunaga and Jiaxuan You and Matei Zaharia and Michael Zhang and Tianyi Zhang and Xikun Zhang and Yuhui Zhang and Lucia Zheng and Kaitlyn Zhou and Percy Liang},
journal= {arXiv preprint arXiv:2108.07258},
year = {2022}
}
备注
Authored by the Center for Research on Foundation Models (CRFM) at the Stanford Institute for Human-Centered Artificial Intelligence (HAI). Report page with citation guidelines: https://crfm.stanford.edu/report.html