English
Related papers

Related papers: Using Large Language Models for Black-Box Testing …

200 papers

Unit testing is a fundamental practice in modern software engineering, with the aim of ensuring the correctness, maintainability, and reliability of individual software components. Very recently, with the advances in Large Language Models…

Software Engineering · Computer Science 2025-06-19 Quanjun Zhang , Chunrong Fang , Siqi Gu , Ye Shang , Zhenyu Chen , Liang Xiao

Automated unit test generation is critical for software quality but traditional structure-driven methods often lack the semantic understanding required to produce realistic inputs and oracles. Large language models (LLMs) address this…

Software Engineering · Computer Science 2026-01-01 Bei Chu , Yang Feng , Kui Liu , Zhaoqiang Guo , Yichi Zhang , Hange Shi , Zifan Nan , Baowen Xu

Future wireless networks are expected to incorporate diverse services that often lack general mathematical models. To address such black-box network management tasks, the large language model (LLM) optimizer framework, which leverages…

Information Theory · Computer Science 2025-07-04 Hoon Lee , Wentao Zhou , Merouane Debbah , Inkyu Lee

In this paper we present the first investigation into the effectiveness of Large Language Models (LLMs) for Failure Mode Classification (FMC). FMC, the task of automatically labelling an observation with a corresponding failure mode code,…

Computation and Language · Computer Science 2023-09-18 Michael Stewart , Melinda Hodkiewicz , Sirui Li

Finite state machines (FSMs) are widely used to manage robot behavior logic, particularly in real-world applications that require a high degree of reliability and structure. However, traditional manual FSM design and modification processes…

Robotics · Computer Science 2024-12-10 Xiangyu Robin Gan , Yuxin Ray Song , Nick Walker , Maya Cakmak

Large language models (LLMs) are recognized as systems that closely mimic aspects of human intelligence. This capability has attracted attention from the social science community, who see the potential in leveraging LLMs to replace human…

Computers and Society · Computer Science 2025-03-04 Qiuejie Xie , Qiming Feng , Tianqi Zhang , Qingqiu Li , Linyi Yang , Yuejie Zhang , Rui Feng , Liang He , Shang Gao , Yue Zhang

Large language models (LLMs) offer emerging opportunities for psychological and behavioral research, but methodological guidance is lacking. This article provides a framework for using LLMs as psychological simulators across two primary…

Computers and Society · Computer Science 2026-04-07 Zhicheng Lin

Large Language Models (LLMs) are one of the most promising developments in the field of artificial intelligence, and the software engineering community has readily noticed their potential role in the software development life-cycle.…

Software Engineering · Computer Science 2026-03-16 Greta Dolcetti , Vincenzo Arceri , Eleonora Iotti , Sergio Maffeis , Agostino Cortesi , Enea Zaffanella

Simulating learner actions helps stress-test open-ended interactive learning environments and prototype new adaptations before deployment. While recent studies show the promise of using large language models (LLMs) for simulating human…

Artificial Intelligence · Computer Science 2024-10-15 Amogh Mannekote , Adam Davies , Jina Kang , Kristy Elizabeth Boyer

Interpretability tools that offer explanations in the form of a dialogue have demonstrated their efficacy in enhancing users' understanding (Slack et al., 2023; Shen et al., 2023), as one-off explanations may fall short in providing…

Computation and Language · Computer Science 2024-04-25 Qianli Wang , Tatiana Anikina , Nils Feldhus , Josef van Genabith , Leonhard Hennig , Sebastian Möller

This report documents the development, test, and application of Large Language Models (LLMs) for automated text analysis, with a specific focus on gambling-like elements in digital games, such as lootboxes. The project aimed not only to…

General Economics · Economics 2024-12-13 Thomas Krause , Steffen Otterbach , Johannes Singer

Undeniably, Large Language Models (LLMs) have stirred an extraordinary wave of innovation in the machine learning research domain, resulting in substantial impact across diverse fields such as reinforcement learning, robotics, and computer…

Machine Learning · Computer Science 2024-05-10 Xingyou Song , Yingtao Tian , Robert Tjarko Lange , Chansoo Lee , Yujin Tang , Yutian Chen

The rapid integration of Large Language Models (LLMs) into software engineering practice is reshaping how software testing activities are performed. LLMs are increasingly used to support software testing. Consequently, software testing…

Software Engineering · Computer Science 2026-03-30 Peng Yang , Yunfeng Zhu , Chao Chang , Shengcheng Yu , Zhenyu Chen , Yong Tang

Black-box Large Language Models (LLMs) provide practical and accessible alternatives to other machine learning methods, as they require minimal labeled data and machine learning expertise to develop solutions for various decision making…

Machine Learning · Computer Science 2025-10-22 Ege Beyazit , KL Navaneet , Prashant Mathur , Roi Blanco , Vidit Bansal , Karim Bouyarmane

As API access becomes a primary interface to large language models (LLMs), users often interact with black-box systems that offer little transparency into the deployed model. To reduce costs or maliciously alter model behaviors, API…

Cryptography and Security · Computer Science 2026-04-10 Xiaoyuan Zhu , Yaowen Ye , Tianyi Qiu , Hanlin Zhu , Sijun Tan , Ajraf Mannan , Jonathan Michala , Raluca Ada Popa , Willie Neiswanger

In second language learning, scenario-based conversation practice is important for language learners to achieve fluency in speaking, but students often lack sufficient opportunities to practice their conversational skills with qualified…

Computation and Language · Computer Science 2024-04-01 Shuyao Xu , Long Qin , Tianyang Chen , Zhenzhou Zha , Bingxue Qiu , Weizhi Wang

To ensure the trustworthiness and interpretability of AI systems, it is essential to align machine learning models with human domain knowledge. This can be a challenging and time-consuming endeavor that requires close communication between…

Machine Learning · Computer Science 2025-04-01 Alexander Koebler , Ingo Thon , Florian Buettner

Unit tests play a key role in ensuring the correctness of software. However, manually creating unit tests is a laborious task, motivating the need for automation. Large Language Models (LLMs) have recently been applied to this problem,…

Software Engineering · Computer Science 2023-12-12 Max Schäfer , Sarah Nadi , Aryaz Eghbali , Frank Tip

We introduce SimulBench, a benchmark designed to evaluate large language models (LLMs) across a diverse collection of creative simulation scenarios, such as acting as a Linux terminal or playing text games with users. While these simulation…

Computation and Language · Computer Science 2024-09-13 Qi Jia , Xiang Yue , Tianyu Zheng , Jie Huang , Bill Yuchen Lin

Large Language Models (LLMs) are demonstrating outstanding potential for tasks such as text generation, summarization, and classification. Given that such models are trained on a humongous amount of online knowledge, we hypothesize that…

Software Engineering · Computer Science 2024-03-18 Jiahui Wu , Chengjie Lu , Aitor Arrieta , Tao Yue , Shaukat Ali