English
Related papers

Related papers: Structured Prompts Improve Evaluation of Language …

200 papers

In large language models (LLM)-based recommendation systems (LLM-RSs), accurately predicting user preferences by leveraging the general knowledge of LLMs is possible without requiring extensive training data. By converting recommendation…

Information Retrieval · Computer Science 2024-12-20 Genki Kusano , Kosuke Akimoto , Kunihiro Takeoka

Large Language Models (LLMs) have shown prominent performance in various downstream tasks and prompt engineering plays a pivotal role in optimizing LLMs' performance. This paper, not only as an overview of current prompt engineering…

Computation and Language · Computer Science 2024-09-18 Haochen Li , Jonathan Leung , Zhiqi Shen

Software systems usually provide numerous configuration options that can affect performance metrics such as execution time, memory usage, binary size, or bitrate. On the one hand, making informed decisions is challenging and requires domain…

System prompts in Large Language Models (LLMs) are predefined directives that guide model behaviour, taking precedence over user inputs in text processing and generation. LLM deployers increasingly use them to ensure consistent responses…

Computers and Society · Computer Science 2025-06-24 Anna Neumann , Elisabeth Kirsten , Muhammad Bilal Zafar , Jatinder Singh

Large language models (LLMs) offer substantial promise for text classification in political science, yet their effectiveness often depends on high-quality prompts and exemplars. To address this, we introduce a three-stage framework that…

Computation and Language · Computer Science 2025-04-08 Menglin Liu , Ge Shi

Large Language Models are transforming software engineering, yet prompt management in practice remains ad hoc, hindering reliability, reuse, and integration into industrial workflows. We present Prompt-with-Me, a practical solution for…

Software Engineering · Computer Science 2025-09-23 Ziyou Li , Agnia Sergeyuk , Maliheh Izadi

Large language models (LLMs) have demonstrated strong performance in a wide-range of language tasks without requiring task-specific fine-tuning. However, they remain prone to hallucinations and inconsistencies, and often struggle with…

Computation and Language · Computer Science 2026-03-27 Matt Pauk , Maria Leonor Pacheco

Most prompt-optimization methods refine a single static template, making them ineffective in complex and dynamic user scenarios. Existing query-dependent approaches rely on unstable textual feedback or black-box reward models, providing…

Artificial Intelligence · Computer Science 2025-11-26 Ke Chen , Yifeng Wang , Hassan Almosapeeh , Haohan Wang

Recently, Large Language Models~(LLMs) such as ChatGPT have showcased remarkable abilities in solving general tasks, demonstrating the potential for applications in recommender systems. To assess how effectively LLMs can be used in…

Information Retrieval · Computer Science 2025-01-17 Lanling Xu , Junjie Zhang , Bingqian Li , Jinpeng Wang , Sheng Chen , Wayne Xin Zhao , Ji-Rong Wen

This study addresses the issues of semantic entanglement, unclear label structure, and insufficient feature representation in few-shot text classification, and proposes an optimization framework based on structured prompts to enhance…

Computation and Language · Computer Science 2026-03-02 Jiasen Zheng , Zijun Zhou , Huajun Zhang , Junjiang Lin , Jingyun Jia , Qi Wang

Much of the success of modern language models depends on finding a suitable prompt to instruct the model. Until now, it has been largely unknown how variations in the linguistic expression of prompts affect these models. This study…

Computation and Language · Computer Science 2026-02-17 Jan Philip Wahle , Terry Ruas , Yang Xu , Bela Gipp

Instruction-tuned Language Models (ILMs) have become essential components of modern AI systems, demonstrating exceptional versatility across natural language and reasoning tasks. Among their most impactful applications is code generation,…

Software Engineering · Computer Science 2026-02-18 Zaiyu Cheng , Antonio Mastropaolo

The rapid advancement of large language models (LLMs) demands robust, unbiased, and scalable evaluation methods. However, human annotations are costly to scale, model-based evaluations are susceptible to stylistic biases, and…

Evaluating pragmatic reasoning in large language models (LLMs) remains challenging because model behavior can vary depending on evaluation methods. Previous studies suggest that prompt-based judgments may diverge from models' internal…

Computation and Language · Computer Science 2026-05-12 Ye-eun Cho

Structured reasoning can improve the inference performance of large language models (LLMs), but it also introduces computational cost and control constraints. When additional reasoning structure helps, and when it instead reduces efficiency…

Machine Learning · Computer Science 2026-04-14 Junyu Guo , Shangding Gu , Ming Jin , Costas Spanos , Javad Lavaei

This research investigates prompt designs of evaluating generated texts using large language models (LLMs). While LLMs are increasingly used for scoring various inputs, creating effective prompts for open-ended text evaluation remains…

Computation and Language · Computer Science 2024-06-28 KuanChao Chu , Yi-Pei Chen , Hideki Nakayama

Structured output from large language models (LLMs) has enhanced efficiency in processing generated information and is increasingly adopted in industrial applications. Prior studies have investigated the impact of structured output on LLMs'…

Computation and Language · Computer Science 2025-12-22 Han Yuan , Yue Zhao , Li Zhang , Wuqiong Luo , Zheng Ma

Prompt learning has become a prevalent strategy for adapting vision-language foundation models to downstream tasks. As large language models (LLMs) have emerged, recent studies have explored the use of category-related descriptions as input…

Computer Vision and Pattern Recognition · Computer Science 2023-12-12 Yubin Wang , Xinyang Jiang , De Cheng , Dongsheng Li , Cairong Zhao

Although pretrained language models (PLMs) can be prompted to perform a wide range of language tasks, it remains an open question how much this ability comes from generalizable linguistic understanding versus surface-level lexical patterns.…

Computation and Language · Computer Science 2023-05-23 Terra Blevins , Hila Gonen , Luke Zettlemoyer

Multimodal Large Language Models (MLLMs) are set to transform how machines process and generate human-like responses by integrating diverse modalities such as text, images, and code. Yet, effectively harnessing their capabilities hinges on…

Artificial Intelligence · Computer Science 2025-04-15 Anwesha Mohanty , Venkatesh Balavadhani Parthasarathy , Arsalan Shahid