English
Related papers

Related papers: Validating AI-Generated Code with Live Programming

200 papers

Generative AI tools hold promise to increase human productivity. This paper presents results from a controlled experiment with GitHub Copilot, an AI pair programmer. Recruited software developers were asked to implement an HTTP server in…

Software Engineering · Computer Science 2023-02-14 Sida Peng , Eirini Kalliamvakou , Peter Cihon , Mert Demirer

Powered by recent advances in code-generating models, AI assistants like Github Copilot promise to change the face of programming forever. But what is this new face of programming? We present the first grounded theory analysis of how…

Human-Computer Interaction · Computer Science 2022-11-01 Shraddha Barke , Michael B. James , Nadia Polikarpova

This paper provides a comprehensive review of the literature concerning the utilization of Natural Language Processing (NLP) techniques, with a particular focus on transformer-based large language models (LLMs) trained using Big Code,…

Software Engineering · Computer Science 2023-07-07 Man Fai Wong , Shangxin Guo , Ching Nam Hang , Siu Wai Ho , Chee Wei Tan

Automatic programming has seen increasing popularity due to the emergence of tools like GitHub Copilot which rely on Large Language Models (LLMs). At the same time, automatically generated code faces challenges during deployment due to…

Software Engineering · Computer Science 2024-05-16 Michael R. Lyu , Baishakhi Ray , Abhik Roychoudhury , Shin Hwei Tan , Patanamon Thongtanunam

The increasing use of large language model (LLM)-powered code generation tools, such as GitHub Copilot, is transforming software engineering practices. This paper investigates how developers validate and repair code generated by Copilot and…

Software Engineering · Computer Science 2024-05-28 Ningzhi Tang , Meng Chen , Zheng Ning , Aakash Bansal , Yu Huang , Collin McMillan , Toby Jia-Jun Li

Code-recommendation systems, such as Copilot and CodeWhisperer, have the potential to improve programmer productivity by suggesting and auto-completing code. However, to fully realize their potential, we must understand how programmers…

Software Engineering · Computer Science 2024-04-23 Hussein Mozannar , Gagan Bansal , Adam Fourney , Eric Horvitz

AI assistants for coding are on the rise. However one of the reasons developers and companies avoid harnessing their full potential is the questionable security of the generated code. This paper first reviews the current state-of-the-art…

Cryptography and Security · Computer Science 2024-03-20 Jakub Res , Ivan Homoliak , Martin Perešíni , Aleš Smrčka , Kamil Malinka , Petr Hanacek

This paper presents an AI-assisted programming tool called Copilot for Xcode for program composition and design to support human software developers. By seamlessly integrating cloud-based Large Language Models (LLM) with Apple's local…

Software Engineering · Computer Science 2023-07-28 Chee Wei Tan , Shangxin Guo , Man Fai Wong , Ching Nam Hang

Proof assistants enable users to develop machine-checked proofs regarding software-related properties. Unfortunately, the interactive nature of these proof assistants imposes most of the proof burden on the user, making formal verification…

Software Engineering · Computer Science 2024-05-08 Pedro Carrott , Nuno Saavedra , Kyle Thompson , Sorin Lerner , João F. Ferreira , Emily First

Low-code programming allows citizen developers to create programs with minimal coding effort, typically via visual (e.g. drag-and-drop) interfaces. In parallel, recent AI-powered tools such as Copilot and ChatGPT generate programs from…

Software Engineering · Computer Science 2023-06-01 Nikitha Rao , Jason Tsay , Kiran Kate , Vincent J. Hellendoorn , Martin Hirzel

GenAI solutions like GitHub Copilot have been shown to increase the productivity of software developers. Yet prior work remains unclear on the quality of code produced and the challenges of maintaining it in software projects. If quality…

General Economics · Economics 2026-01-29 Feiyang Xu , Poonacha K. Medappa , Murat M. Tunc , Martijn Vroegindeweij , Jan C. Fransoo

Large language models (LLMs) are increasingly used in software development, generating code that ranges from short snippets to substantial project components. As AI-generated code becomes more common in real-world repositories, it is…

Software Engineering · Computer Science 2026-04-06 Tianhao Mao , Dongfang Zhao , Haixu Tang , Xiaofeng Wang , Hang Zhang

The integration of Large Language Models (LLMs) such as GitHub Copilot, ChatGPT, Cursor AI, and Codeium AI into software development has revolutionized the coding landscape, offering significant productivity gains, automation, and enhanced…

AI-powered code assistants, such as Copilot, are quickly becoming a ubiquitous component of contemporary coding contexts. Among these environments, computational notebooks, such as Jupyter, are of particular interest as they provide rich…

Human-Computer Interaction · Computer Science 2023-01-27 Andrew M. McNutt , Chenglong Wang , Robert A. DeLine , Steven M. Drucker

Data analysis is challenging as it requires synthesizing domain knowledge, statistical expertise, and programming skills. Assistants powered by large language models (LLMs), such as ChatGPT, can assist analysts by translating natural…

Human-Computer Interaction · Computer Science 2024-03-05 Ken Gu , Ruoxi Shang , Tim Althoff , Chenglong Wang , Steven M. Drucker

In software development there is a constant pressure to produce code faster and faster without compromising on quality. New tools supporting developers are created in response to this demand. Currently a new generation of such solutions is…

Software Engineering · Computer Science 2023-01-13 Mateusz Jaworski , Dariusz Piotrkowski

Motivation. Trust in generative AI programming assistants is a vital attitude that impacts how programmers use those programming assistants. Programmers that are over-trusting may be too reliant on their tools, leading to incorrect or…

Human-Computer Interaction · Computer Science 2025-09-17 Anshul Shah , Thomas Rexin , Elena Tomson , Leo Porter , William G. Griswold , Adalbert Gerald Soosai Raj

The rise of large language models (LLMs) has accelerated the development of automated techniques and tools for supporting various software engineering tasks, e.g., program understanding, code generation, software testing, and program…

Software Engineering · Computer Science 2026-01-29 Maja Vukovic , Rangeet Pan , Tin Kam Ho , Rahul Krishna , Raju Pavuluri , Michele Merler

The study on GitHub Copilot by GovTech Singapore's Engineering Productivity Programme (EPP) reveals significant potential for AI Code Assistant tools to boost developer productivity and improve application quality in the public sector.…

Software Engineering · Computer Science 2024-09-27 Kevin KB Ng , Liyana Fauzi , Leon Leow , Jaren Ng

The rapid adoption of Artificial Intelligence(AI) programming assistants such as GitHub Copilot introduces new challenges in how these software tools address human needs. Many existing evaluation frameworks address technical aspects such as…

Software Engineering · Computer Science 2025-08-07 Soroush Heydari