ChatGPT 能否获得工程学位？评估 AI 助手对高等教育的脆弱性
计算机与社会
2024-11-28 v2 人工智能
计算与语言
摘要
AI 助手正越来越多地被学生在高等教育机构中使用。尽管这些工具为改进教学和教育提供了机会，但也对评估和学习结果提出了重大挑战。我们通过脆弱性视角来概念化这些挑战，即大学评估和学习结果可能受到学生使用生成式 AI 的影响的程度。我们通过衡量 AI 助手完成高等教育机构中标准 STEM 课程评估问题的程度来检验这种脆弱性的潜在规模。具体而言，我们编译了一个来自 EPFL 50 门课程的文本评估问题的新型数据集，并评估两种 AI 助手，GPT-3.5 和 GPT-4 能否充分回答这些问题。我们使用八种提示策略来生成响应，并发现 GPT-4 在平均 65.8% 的问题上回答正确，并且即使在不同提示策略中，GPT-4 也能在 85.1% 的问题上产生正确答案。当按我们数据集中的课程按学位项目分组时，这些系统已经通过了大量核心课程在各种学位项目中的非项目评估，构成了高等教育 accreditation 的风险，随着这些模型的不断改进，这种风险将被放大。我们的结果呼吁在生成式 AI 的进步中，重新考虑高等教育中的程序层面评估设计。
引用
@article{arxiv.2408.11841,
title = {Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants},
author = {Beatriz Borges and Negar Foroutan and Deniz Bayazit and Anna Sotnikova and Syrielle Montariol and Tanya Nazaretzky and Mohammadreza Banaei and Alireza Sakhaeirad and Philippe Servant and Seyed Parsa Neshaei and Jibril Frej and Angelika Romanou and Gail Weiss and Sepideh Mamooler and Zeming Chen and Simin Fan and Silin Gao and Mete Ismayilzada and Debjit Paul and Alexandre Schöpfer and Andrej Janchevski and Anja Tiede and Clarence Linden and Emanuele Troiani and Francesco Salvi and Freya Behrens and Giacomo Orsi and Giovanni Piccioli and Hadrien Sevel and Louis Coulon and Manuela Pineros-Rodriguez and Marin Bonnassies and Pierre Hellich and Puck van Gerwen and Sankalp Gambhir and Solal Pirelli and Thomas Blanchard and Timothée Callens and Toni Abi Aoun and Yannick Calvino Alonso and Yuri Cho and Alberto Chiappa and Antonio Sclocchi and Étienne Bruno and Florian Hofhammer and Gabriel Pescia and Geovani Rizk and Leello Dadi and Lucas Stoffl and Manoel Horta Ribeiro and Matthieu Bovel and Yueyang Pan and Aleksandra Radenovic and Alexandre Alahi and Alexander Mathis and Anne-Florence Bitbol and Boi Faltings and Cécile Hébert and Devis Tuia and François Maréchal and George Candea and Giuseppe Carleo and Jean-Cédric Chappelier and Nicolas Flammarion and Jean-Marie Fürbringer and Jean-Philippe Pellet and Karl Aberer and Lenka Zdeborová and Marcel Salathé and Martin Jaggi and Martin Rajman and Mathias Payer and Matthieu Wyart and Michael Gastpar and Michele Ceriotti and Ola Svensson and Olivier Lévêque and Paolo Ienne and Rachid Guerraoui and Robert West and Sanidhya Kashyap and Valerio Piazza and Viesturs Simanis and Viktor Kuncak and Volkan Cevher and Philippe Schwaller and Sacha Friedli and Patrick Jermann and Tanja Käser and Antoine Bosselut},
journal= {arXiv preprint arXiv:2408.11841},
year = {2024}
}
备注
20 pages, 8 figures