English
Related papers

Related papers: Using LLMs to Evaluate Architecture Documents: Res…

200 papers

Automatic reviewing helps handle a large volume of papers, provides early feedback and quality control, reduces bias, and allows the analysis of trends. We evaluate the alignment of automatic paper reviews with human reviews using an arena…

With the rapid advancement of AI, software engineering increasingly relies on AI-driven approaches, particularly language models (LMs), to enhance code performance. However, the trustworthiness and reliability of LMs remain significant…

Software Engineering · Computer Science 2025-03-19 Jingzhi Gong

The exponential growth of text-based data in domains such as healthcare, education, and social sciences has outpaced the capacity of traditional qualitative analysis methods, which are time-intensive and prone to subjectivity. Large…

Recently, Large Language Models (LLMs) have demonstrated significant potential in automating software engineering tasks. Generating software architecture designs from requirement documents is a crucial step in software development. However,…

Software Engineering · Computer Science 2026-04-09 Minxiao Li , Shuying Yan , Li Zhang , Yang Liu , Fang Liu

Large language models (LLMs) have achieved remarkable performance in language understanding and generation tasks by leveraging vast amounts of online texts. Unlike conventional models, LLMs can adapt to new domains through prompt…

Artificial Intelligence · Computer Science 2024-06-18 Ming Cheung

LLM-based agents represent a paradigm shift in AI, enabling autonomous systems to plan, reason, and use tools while interacting with dynamic environments. This paper provides the first comprehensive survey of evaluation methods for these…

Artificial Intelligence · Computer Science 2026-04-24 Asaf Yehudai , Lilach Eden , Alan Li , Guy Uziel , Yilun Zhao , Roy Bar-Haim , Arman Cohan , Michal Shmueli-Scheuer

Artificial Intelligence (AI) techniques, especially Large Language Models (LLMs), have started gaining popularity among researchers and software developers for generating source code. However, LLMs have been shown to generate code with…

Software Engineering · Computer Science 2024-11-08 Hyunjae Suh , Mahan Tafreshipour , Jiawei Li , Adithya Bhattiprolu , Iftekhar Ahmed

Algorithm design is crucial for effective problem-solving across various domains. The advent of Large Language Models (LLMs) has notably enhanced the automation and innovation within this field, offering new perspectives and promising…

Machine Learning · Computer Science 2026-01-06 Fei Liu , Yiming Yao , Ping Guo , Zhiyuan Yang , Zhe Zhao , Xi Lin , Xialiang Tong , Kun Mao , Zhichao Lu , Zhenkun Wang , Mingxuan Yuan , Qingfu Zhang

Large Language Models (LLMs) are increasingly being integrated into software development processes, with the potential to transform team workflows and productivity. This paper investigates how LLMs affect team collaboration throughout the…

Software Engineering · Computer Science 2025-10-13 Devang Dhanuka

It is expected that in the near future, AI software development assistants will play an important role in the software industry. However, current software development assistants tend to be unreliable, often producing incorrect, unsafe, or…

Software Engineering · Computer Science 2024-01-24 Daniel Maninger , Krishna Narasimhan , Mira Mezini

Modern software systems are subjected to various types of uncertainties arising from context, environment, etc. To this end, self-adaptation techniques have been sought out as potential solutions. Although recent advances in self-adaptation…

Software Engineering · Computer Science 2024-04-16 Raghav Donakanti , Prakhar Jain , Shubham Kulkarni , Karthik Vaidhyanathan

The use of large language models (LLMs) in qualitative analysis offers enhanced efficiency but raises questions about their alignment with the contextual nature of research for design (RfD). This research examines the trustworthiness of…

Human-Computer Interaction · Computer Science 2025-04-24 Joel Oksanen , Andrés Lucero , Perttu Hämäläinen

Large Language Models (LLMs) are widely used in software engineering to generate, complete, translate, and fix code, improving developer productivity. While most research focuses on the energy consumption and carbon emissions of model…

Software Engineering · Computer Science 2026-04-15 Sabiya Banu Masthan Ali , Oussema Kirmani , Aroosa Hameed , Syed Muhammad Danish , Gautam Srivastava

Large language models (LLMs) are now an integral part of software development workflows and are reshaping the whole process. Traditional technology stack selection has not caught up. Most of the existing selection methods focus solely on…

Software Engineering · Computer Science 2025-09-16 Xiaoyu Zhang , Weipeng Jiang , Juan Zhai , Shiqing Ma , Qingshuang Bao , Chenhao Lin , Chao Shen , Tianlin Li , Yang Liu

Large language models (LLMs) have shown significant potential to change how we write, communicate, and create, leading to rapid adoption across society. This dissertation examines how individuals and institutions are adapting to and…

Computation and Language · Computer Science 2025-06-24 Weixin Liang

While the use of Large Language Models (LLMs) in programming has been extensively studied, there is limited understanding of how LLMs support collaborative work where creativity plays a central role. Software design, as a collaborative and…

Software Engineering · Computer Science 2026-04-28 Victoria Jackson , Grischa Liebel , Rafael Prikladnicki , Andre van der Hoek

A paradigm shift is underway in Software Engineering, with AI systems such as LLMs playing an increasingly important role in boosting software development productivity. This trend is anticipated to persist. In the next years, we expect a…

Software Engineering · Computer Science 2026-03-27 Valerio Terragni , Annie Vella , Partha Roop , Kelly Blincoe

As Large Language Models (LLMs) and other forms of Generative AI permeate various aspects of our lives, their application for learning and education has provided opportunities and challenges. This paper presents an investigation into the…

Computers and Society · Computer Science 2023-06-06 Daniel Leiker , Sara Finnigan , Ashley Ricker Gyllen , Mutlu Cukurova

Research within sociotechnical domains, such as Software Engineering, fundamentally requires a thorough consideration of the human perspective. However, traditional qualitative data collection methods suffer from challenges related to…

Software Engineering · Computer Science 2023-11-21 Marco A. Gerosa , Bianca Trinkenreich , Igor Steinmacher , Anita Sarma

LLM-based agents are becoming central to software engineering tasks, yet evaluating them remains fragmented and largely model-centric. Existing studies overlook how architectural components, such as planners, memory, and tool routers, shape…

Software Engineering · Computer Science 2026-01-28 Débora Souza , Patrícia Machado