English
Related papers

Related papers: Automatic Program Repair with OpenAI's Codex: Eval…

200 papers

Large language models (LLMs), such as OpenAI's Codex, have demonstrated their potential to generate code from natural language descriptions across a wide range of programming tasks. Several benchmarks have recently emerged to evaluate the…

Software Engineering · Computer Science 2023-04-11 Sarah Fakhoury , Saikat Chakraborty , Madan Musuvathi , Shuvendu K. Lahiri

Techniques of Automatic Program Repair (APR) have the potential of thoroughly facilitating the task of producing quality software. After a promising start, however, progress in making APR practical has been hindered by the lack of a common…

Software Engineering · Computer Science 2024-02-23 Bertrand Meyer , Viktoryia Kananchuk , Li Huang

Modern automated program repair (APR) is well-tuned to finding and repairing bugs that introduce observable erroneous behavior to a program. However, a significant class of bugs does not lead to such observable behavior (e.g.,…

Software Engineering · Computer Science 2022-11-09 Omar I. Al-Bataineh , Leon Moonen

Machine learning-based program analyses have recently shown the promise of integrating formal and probabilistic reasoning towards aiding software development. However, in the absence of large annotated corpora, training these analyses is…

Machine Learning · Computer Science 2021-11-17 Miltiadis Allamanis , Henry Jackson-Flux , Marc Brockschmidt

Test-based automated program repair has been a prolific field of research in software engineering in the last decade. Many approaches have indeed been proposed, which leverage test suites as a weak, but affordable, approximation to program…

Among areas of software engineering where AI techniques -- particularly, Large Language Models -- seem poised to yield dramatic improvements, an attractive candidate is Automatic Program Repair (APR), the production of satisfactory…

Software Engineering · Computer Science 2025-08-05 Li Huang , Ilgiz Mustafin , Marco Piccioni , Alessandro Schena , Reto Weber , Bertrand Meyer

Automatic Program Repair (APR) endeavors to autonomously rectify issues within specific projects, which generally encompasses three categories of tasks: bug resolution, new feature development, and feature enhancement. Despite extensive…

Software Engineering · Computer Science 2024-09-24 Jiuang Zhao , Donghao Yang , Li Zhang , Xiaoli Lian , Zitian Yang , Fang Liu

Automatic Program Repair (APR) techniques can promisingly help reducing the cost of debugging. Many relevant APR techniques follow the generate-and-validate approach, that is, the faulty program is iteratively modified with different change…

Software Engineering · Computer Science 2022-05-06 Davide Ginelli , Matias Martinez , Leonardo Mariani , Martin Monperrus

In the past decade, research on test-suite-based automatic program repair has grown significantly. Each year, new approaches and implementations are featured in major software engineering venues. However, most of those approaches are…

Software Engineering · Computer Science 2019-05-29 Thomas Durieux , Fernanda Madeiral , Matias Martinez , Rui Abreu

In this study, we evaluated the capability of Large Language Models (LLMs), particularly OpenAI's GPT-4, in detecting software vulnerabilities, comparing their performance against traditional static code analyzers like Snyk and Fortify. Our…

Software Engineering · Computer Science 2023-08-22 David Noever

We solve university level probability and statistics questions by program synthesis using OpenAI's Codex, a Transformer trained on text and fine-tuned on code. We transform course problems from MIT's 18.05 Introduction to Probability and…

Machine Learning · Computer Science 2021-11-17 Leonard Tang , Elizabeth Ke , Nikhil Singh , Nakul Verma , Iddo Drori

Bug fixing and code generation have been core research topics in software development for many years. The recent explosive growth in Large Language Models has completely transformed these spaces, putting in reach incredibly powerful tools…

Artificial Intelligence · Computer Science 2024-11-13 Avinash Anand , Akshit Gupta , Nishchay Yadav , Shaurya Bajaj

Large language models (LMs) of code have recently shown tremendous promise in completing code and synthesizing code from natural language descriptions. However, the current state-of-the-art code LMs (e.g., Codex (Chen et al., 2021)) are not…

Programming Languages · Computer Science 2022-05-05 Frank F. Xu , Uri Alon , Graham Neubig , Vincent J. Hellendoorn

Automated Program Repair (APR) improves developer productivity by saving debugging and bug-fixing time. While APR has been extensively explored for C/C++ and Java programs, there is little research on bugs in PHP programs due to the lack of…

Software Engineering · Computer Science 2024-01-23 K. D. Pramod , W. T. N. De Silva , W. U. K. Thabrew , Ridwan Shariffdeen , Sandareka Wickramanayake

Current approaches to program synthesis with Large Language Models (LLMs) exhibit a "near miss syndrome": they tend to generate programs that semantically resemble the correct answer (as measured by text similarity metrics or human…

Software Engineering · Computer Science 2023-04-21 Vadim Liventsev , Anastasiia Grishina , Aki Härmä , Leon Moonen

Automated program repair (APR) has attracted widespread attention in recent years with substantial techniques being proposed. Meanwhile, a number of benchmarks have been established for evaluating the performances of APR techniques, among…

Software Engineering · Computer Science 2019-01-09 Shangwen Wang , Ming Wen , Xiaoguang Mao , Deheng Yang

At ICSE'2013, there was the first session ever dedicated to automatic program repair. In this session, Kim et al. presented PAR, a novel template-based approach for fixing Java bugs. We strongly disagree with key points of this paper. Our…

Software Engineering · Computer Science 2022-02-11 Martin Monperrus

Automated program repair is an emerging technology that seeks to automatically rectify bugs and vulnerabilities using learning, search, and semantic analysis. Trust in automatically generated patches is necessary for achieving greater…

Software Engineering · Computer Science 2022-02-14 Yannic Noller , Ridwan Shariffdeen , Xiang Gao , Abhik Roychoudhury

GitHub Copilot is an artificial intelligence model for automatically generating source code from natural language problem descriptions. Since June 2022, Copilot has officially been available for free to all students as a plug-in to…

Human-Computer Interaction · Computer Science 2022-10-28 Paul Denny , Viraj Kumar , Nasser Giacaman

Software vulnerabilities pose significant security threats, requiring effective mitigation. While Automated Program Repair (APR) has advanced in fixing general bugs, vulnerability patching, a security-critical aspect of APR remains…

Software Engineering · Computer Science 2025-06-06 Zanis Ali Khan , Aayush Garg , Qiang Tang