English
Related papers

Related papers: Evaluating the Code Quality of AI-Assisted Code Ge…

200 papers

ChatGPT and Bard are AI chatbots based on Large Language Models (LLM) that are slated to promise different applications in diverse areas. In education, these AI technologies have been tested for applications in assessment and teaching. In…

Computation and Language · Computer Science 2023-05-11 Abdolvahab Khademi

The use of AI code-generation tools is becoming increasingly common, making it important to understand how software developers are adopting these tools. In this study, we investigate how developers engage with Amazon's CodeWhisperer, an…

Software Engineering · Computer Science 2025-10-14 Jeena Javahar , Tanya Budhrani , Manaal Basha , Cleidson R. B. de Souza , Ivan Beschastnikh , Gema Rodriguez-Perez

GitHub Copilot, an extension for the Visual Studio Code development environment powered by the large-scale language model Codex, makes automatic program synthesis available for software developers. This model has been extensively studied in…

Software Engineering · Computer Science 2021-11-16 Dominik Sobania , Martin Briesch , Franz Rothlauf

The emergence of ChatGPT has recently garnered significant attention from the computational linguistics community. To demonstrate its capabilities as a keyphrase generator, we conduct a preliminary evaluation of ChatGPT for the keyphrase…

Computation and Language · Computer Science 2023-12-25 Mingyang Song , Haiyun Jiang , Shuming Shi , Songfang Yao , Shilong Lu , Yi Feng , Huafeng Liu , Liping Jing

Artificial Intelligence (AI) tools such as GitHub Copilot and ChatGPT are increasingly used in software engineering (SE) for tasks such as code, test, and documentation generation. However, engineers often face uncertainty about when to…

Software Engineering · Computer Science 2026-03-24 Vahid Garousi , Zafar Jafarov , Aytan Mövsümova , Atif Namazov

ChatGPT is a natural language processing tool that can engage in human-like conversations and generate coherent and contextually relevant responses to various prompts. ChatGPT is capable of understanding natural text that is input by a user…

Human-Computer Interaction · Computer Science 2024-09-02 Sara Amani , Lance White , Trini Balart , Laksha Arora , Kristi J. Shryock , Kelly Brumbelow , Karan L. Watson

Large Code Models (LCMs) show potential in code intelligence, but their effectiveness is greatly influenced by prompt quality. Current prompt design is mostly manual, which is time-consuming and highly dependent on specific LCMs and tasks.…

Software Engineering · Computer Science 2025-11-06 Kexing Ji , Shiyun Fu , Cuiyun Gao , Yujia Chen , Zezhou Yang , Chaozheng Wang , Yuetang Deng

In recent years, artificial intelligence has had a conspicuous growth in almost every aspect of life. One of the most applicable areas is security code review, in which a lot of AI-based tools and approaches have been proposed. Recently,…

Cryptography and Security · Computer Science 2023-08-29 Atieh Bakhshandeh , Abdalsamad Keramatfar , Amir Norouzi , Mohammad Mahdi Chekidehkhoun

When graduates of computing degree programs enter the software industry, they will most likely join teams working on legacy code bases developed by people other than themselves. In these so-called brownfield software development settings,…

Software Engineering · Computer Science 2025-06-13 Md Istiak Hossain Shihab , Christopher Hundhausen , Ahsun Tariq , Summit Haque , Yunhan Qiao , Brian Mulanda

The use of generative AI-based coding assistants like ChatGPT and Github Copilot is a reality in contemporary software development. Many of these tools are provided as remote APIs. Using third-party APIs raises data privacy and security…

Software Engineering · Computer Science 2025-01-20 Negar Alizadeh , Boris Belchev , Nishant Saurabh , Patricia Kelbert , Fernando Castor

The increasing demand for digital literacy and artificial intelligence (AI) fluency in the workforce has highlighted the need for scalable, efficient programming instruction. This study evaluates the effectiveness of integrating generative…

Computers and Society · Computer Science 2025-05-28 Ian McCulloh , Pedro Rodriguez , Srivaths Kumar , Manu Gupta , Viplove Raj Sharma , Benjamin Johnson , Anthony N. Johnson

The software engineering community recently has witnessed widespread deployment of AI programming assistants, such as GitHub Copilot. However, in practice, developers do not accept AI programming assistants' initial suggestions at a high…

Software Engineering · Computer Science 2023-09-19 Jenny T. Liang , Chenyang Yang , Brad A. Myers

The introduction of large language models (LLMs) like ChatGPT and Google Palm2 for smart contract generation seems to be the first well-established instance of an AI pair programmer. LLMs have access to a large number of open-source smart…

Software Engineering · Computer Science 2023-08-22 Rabimba Karanjai , Edward Li , Lei Xu , Weidong Shi

This study explores the integration and impact of ChatGPT, a generative AI that utilizes natural language processing, in an educational environment. The main goal is to evaluate how ChatGPT affects project performance. To this end, we…

Computers and Society · Computer Science 2024-09-09 Sunhee Hwang , Yudoo Kim , Heejin Lee

ChatGPT and other large language models (LLMs) promise to revolutionize software development by automatically generating code from program specifications. We assess the performance of ChatGPT's GPT-3.5-turbo model on LeetCode, a popular…

Software Engineering · Computer Science 2024-11-13 Minda Li , Bhaskar Krishnamachari

The integration of Large Language Models (LLMs) such as GitHub Copilot, ChatGPT, Cursor AI, and Codeium AI into software development has revolutionized the coding landscape, offering significant productivity gains, automation, and enhanced…

Generative AI technologies such as ChatGPT, Gemini, and MidJourney have made remarkable progress in recent years. Recent literature has documented ChatGPT's positive impact on productivity in areas where it has strong expertise,…

General Economics · Economics 2024-03-05 Voraprapa Nakavachara , Tanapong Potipiti , Thanee Chaiwat

Pre-trained code models have emerged as crucial tools in various code intelligence tasks. However, their effectiveness depends on the quality of the pre-training dataset, particularly the human reference comments, which serve as a bridge…

Software Engineering · Computer Science 2023-12-27 Kang Yang , Xinjun Mao , Shangwen Wang , Tanghaoran Zhang , Bo Lin , Yanlin Wang , Yihao Qin , Zhang Zhang , Xiaoguang Mao

Learning new programming skills requires tailored guidance. With the emergence of advanced Natural Language Generation models like the ChatGPT API, there is now a possibility of creating a convenient and personalized tutoring system with AI…

Human-Computer Interaction · Computer Science 2023-06-16 Eason Chen , Ray Huang , Han-Shin Chen , Yuen-Hsien Tseng , Liang-Yi Li

Large language models (LLMs), such as ChatGPT released by OpenAI, have attracted significant attention from both industry and academia due to their demonstrated ability to generate high-quality content for various tasks. Despite the…

Software Engineering · Computer Science 2024-11-08 Xiaodan Xu , Chao Ni , Xinrong Guo , Shaoxuan Liu , Xiaoya Wang , Kui Liu , Xiaohu Yang