English
Related papers

Related papers: Investigating the Efficacy of Large Language Model…

200 papers

Without writing a single line of code by a human, an example Monte Carlo simulation based application for stochastic dependence modeling with copulas is developed using a state-of-the-art large language model (LLM) fine-tuned for…

Computation and Language · Computer Science 2023-04-03 Jan Górecki

Leveraging large language models (LLMs) for various natural language processing tasks has led to superlative claims about their performance. For the evaluation of machine translation (MT), existing research shows that LLMs are able to…

Computation and Language · Computer Science 2024-10-10 Shenbin Qian , Archchana Sindhujan , Minnie Kabra , Diptesh Kanojia , Constantin Orăsan , Tharindu Ranasinghe , Frédéric Blain

This paper investigates the performance of the Large Language Models (LLMs) ChatGPT-3.5 and GPT-4 in solving introductory programming tasks. Based on the performance, implications for didactic scenarios and assessment formats utilizing LLMs…

Software Engineering · Computer Science 2023-08-21 Natalie Kiesler , Daniel Schiffner

This paper explores the use of Large Language Models (LLMs) and in particular ChatGPT in programming, source code analysis, and code generation. LLMs and ChatGPT are built using machine learning and artificial intelligence techniques, and…

Software Engineering · Computer Science 2023-06-12 Ahmed R. Sadik , Antonello Ceravola , Frank Joublin , Jibesh Patra

Recently, large language models (LLM) based generative AI has been gaining momentum for their impressive high-quality performances in multiple domains, particularly after the release of the ChatGPT. Many believe that they have the potential…

Software Engineering · Computer Science 2024-02-22 Wei Wang , Huilong Ning , Gaowei Zhang , Libo Liu , Yi Wang

Large language models (LLMs) such as GPT-3.5 and CodeLlama are powerful models for code generation and understanding. Fine-tuning these models comes with a high computational cost and requires a large labeled dataset. Alternatively,…

Software Engineering · Computer Science 2024-01-30 Kamel Alrashedy , Ahmed Binjahlan

Large Language Models (LLMs) have recently shown promise as high-level planners for robots when given access to a selection of low-level skills. However, it is often assumed that LLMs do not possess sufficient knowledge to be used for the…

Robotics · Computer Science 2024-06-19 Teyun Kwon , Norman Di Palo , Edward Johns

In recent years, large language models (LLMs) have emerged as powerful tools with potential applications in various fields, including software engineering. Within the scope of this research, we evaluate five different state-of-the-art LLMs…

Computation and Language · Computer Science 2024-09-09 Luis Mayer , Christian Heumann , Matthias Aßenmacher

Despite being introduced only a few years ago, Large Language Models (LLMs) are already widely used by developers for code generation. However, their application in automating other Software Engineering activities remains largely…

Software Engineering · Computer Science 2024-09-27 Aylton Almeida , Laerte Xavier , Marco Tulio Valente

The rapid advancements in large language models (LLMs) have greatly expanded the potential for automated code-related tasks. Two primary methodologies are used in this domain: prompt engineering and fine-tuning. Prompt engineering involves…

Software Engineering · Computer Science 2025-02-21 Jiho Shin , Clark Tang , Tahmineh Mohati , Maleknaz Nayebi , Song Wang , Hadi Hemmati

Large language models (LLMs) like ChatGPT (i.e., gpt-3.5-turbo and gpt-4) exhibited remarkable advancement in a range of software engineering tasks associated with source code such as code review and code generation. In this paper, we…

Software Engineering · Computer Science 2023-10-17 Michael Fu , Chakkrit Tantithamthavorn , Van Nguyen , Trung Le

With the involvement of multiple programming languages in modern software development, cross-lingual code clone detection has gained traction within the software engineering community. Numerous studies have explored this topic, proposing…

Software Engineering · Computer Science 2025-05-07 Micheline Bénédicte Moumoula , Abdoul Kader Kabore , Jacques Klein , Tegawendé Bissyande

As large language models (LLMs) become more common in educational tools and programming environments, questions arise about how these systems should interact with users. This study investigates how different interaction styles with…

Human-Computer Interaction · Computer Science 2025-07-08 Kai Deng

Large Language Models (LLMs) have demonstrated promise in medical knowledge assessments, yet their practical utility in real-world clinical decision-making remains underexplored. In this study, we evaluated the performance of three…

Computation and Language · Computer Science 2025-12-30 Mengdi Chai , Ali R. Zomorrodi

This study investigates the reliability of code generation by Large Language Models (LLMs), focusing on identifying and analyzing defects in the generated code. Despite the advanced capabilities of LLMs in automating code generation,…

Software Engineering · Computer Science 2024-08-27 Ali Mohammadi Esfahani , Nafiseh Kahani , Samuel A. Ajila

Over the last few years, large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP) that fundamentally transform research and developments in the field. ChatGPT represents one of the…

Computation and Language · Computer Science 2023-04-13 Viet Dac Lai , Nghia Trung Ngo , Amir Pouran Ben Veyseh , Hieu Man , Franck Dernoncourt , Trung Bui , Thien Huu Nguyen

With the increase in software vulnerabilities that cause significant economic and social losses, automatic vulnerability detection has become essential in software development and maintenance. Recently, large language models (LLMs) like GPT…

Software Engineering · Computer Science 2024-04-15 Chenyuan Zhang , Hao Liu , Jiutian Zeng , Kejing Yang , Yuhong Li , Hui Li

Large language models (LLMs) have achieved state-of-the-art performance in various software engineering tasks, including error detection, clone detection, and code translation, primarily leveraging high-resource programming languages like…

Computation and Language · Computer Science 2025-06-11 Razan Baltaji , Saurabh Pujar , Louis Mandel , Martin Hirzel , Luca Buratti , Lav Varshney

Multilingual programming, which involves using multiple programming languages (PLs) in a single project, is increasingly common due to its benefits. However, it introduces cross-language bugs (CLBs), which arise from interactions between…

Software Engineering · Computer Science 2026-04-22 Zengyang Li , Yimeng Li , Binbin Huang , Peng Liang , Ran Mo , Hui Liu , Yutao Ma

Instruction-tuned Language Models ILMs have become essential components of modern AI systems, demonstrating exceptional versatility across a wide range of natural language and reasoning tasks. Among their most impactful applications is code…

Software Engineering · Computer Science 2026-02-19 Saima Afrin , Zaiyu Cheng , Tushar Sharma , Alexander Serebrenik , Massimiliano Di Penta , Antonio Mastropaolo