English
Related papers

Related papers: PromptInfuser: How Tightly Coupling AI and UI Desi…

200 papers

Interaction with Large Language Models (LLMs) is primarily carried out via prompting. A prompt is a natural language instruction designed to elicit certain behaviour or output from a model. In theory, natural language prompts enable…

Human-Computer Interaction · Computer Science 2024-03-15 Michael Desmond , Michelle Brachman

We introduce ComfyUI-Copilot, a large language model-powered plugin designed to enhance the usability and efficiency of ComfyUI, an open-source platform for AI-driven art creation. Despite its flexibility and user-friendly interface,…

Computation and Language · Computer Science 2025-06-06 Zhenran Xu , Xue Yang , Yiyu Wang , Qingli Hu , Zijiao Wu , Longyue Wang , Weihua Luo , Kaifu Zhang , Baotian Hu , Min Zhang

Recent advancements in AI foundation models have made it possible for them to be utilized off-the-shelf for creative tasks, including ideating design concepts or generating visual prototypes. However, integrating these models into the…

Human-Computer Interaction · Computer Science 2024-06-26 David Chuan-En Lin , Nikolas Martelaro

In automated UI design generation, a key challenge is the lack of support for iterative processes, as most systems focus solely on end-to-end output. This stems from limited capabilities in interpreting design intent and a lack of…

Human-Computer Interaction · Computer Science 2025-10-30 Mingyue Yuan , Jieshan Chen , Yongquan Hu , Sidong Feng , Mulong Xie , Gelareh Mohammadi , Zhenchang Xing , Aaron Quigley

Generative AI has enabled novice designers to quickly create professional-looking visual representations for product concepts. However, novices have limited domain knowledge that could constrain their ability to write prompts that…

Human-Computer Interaction · Computer Science 2026-03-30 Sirui Tao , Ivan Liang , Cindy Peng , Zhiqing Wang , Srishti Palani , Steven P. Dow

Recent technological advances popularized the use of image generation among the general public. Crafting effective prompts can, however, be difficult for novice users. To tackle this challenge, we developed PromptMap, a new interaction…

Human-Computer Interaction · Computer Science 2025-04-04 Krzysztof Adamkiewicz , Paweł W. Woźniak , Julia Dominiak , Andrzej Romanowski , Jakob Karolus , Stanislav Frolov

Generative AI, such as image generation models and large language models, stands to provide tremendous value to end-user programmers in creative and knowledge workflows. Current research methods struggle to engage end-users in a realistic…

Human-Computer Interaction · Computer Science 2023-12-29 Advait Sarkar , Ian Drosos , Rob Deline , Andrew D. Gordon , Carina Negreanu , Sean Rintel , Jack Williams , Benjamin Zorn

The widespread adoption of large language models (LLMs) such as ChatGPT, Gemini, and DeepSeek has significantly changed how people approach tasks in education, professional work, and creative domains. This paper investigates how the…

Human-Computer Interaction · Computer Science 2025-08-29 Rizal Khoirul Anam

Text entry is an essential task in our day-to-day digital interactions. Numerous intelligent features have been developed to streamline this process, making text entry more effective, efficient, and fluid. These improvements include…

Computation and Language · Computer Science 2023-10-17 Junxiao Shen , John J. Dudley , Jingyao Zheng , Bill Byrne , Per Ola Kristensson

The rapid development of artificial intelligence (AI) has fundamentally transformed creative work practices in the design industry. Existing studies have identified both opportunities and challenges for creative practitioners in their…

Human-Computer Interaction · Computer Science 2025-09-30 Nami Ogawa , Yuki Okafuji , Yuji Hatada , Jun Baba

This paper studies how large language models (LLMs) can act as effective, high-level creative collaborators and ``muses'' for game design. We model the design of this study after the exercises artists use by looking at amorphous ink…

Artificial Intelligence · Computer Science 2024-03-06 Asad Anjum , Yuting Li , Noelle Law , M Charity , Julian Togelius

Text-to-image AI are capable of generating novel images for inspiration, but their applications for 3D design workflows and how designers can build 3D models using AI-provided inspiration have not yet been explored. To investigate this, we…

Human-Computer Interaction · Computer Science 2023-08-02 Vivian Liu , Jo Vermeulen , George Fitzmaurice , Justin Matejka

While LLMs can effectively help prototype single ML functionalities, many real-world applications involve complex tasks that cannot be easily handled via a single run of an LLM. Recent work has found that chaining multiple LLM runs together…

Human-Computer Interaction · Computer Science 2022-03-15 Tongshuang Wu , Ellen Jiang , Aaron Donsbach , Jeff Gray , Alejandra Molina , Michael Terry , Carrie J Cai

Large Language Models (LLMs) have transformed human-computer interaction by enabling natural language-based communication with AI-powered chatbots. These models are designed to be intuitive and user-friendly, allowing users to articulate…

Generative AI is changing the way developers interact with software systems, providing services that can produce and deliver new content, crafted to satisfy the actual needs of developers. For instance, developers can ask for new code…

Software Engineering · Computer Science 2024-02-15 Ionut Daniel Fagadau , Leonardo Mariani , Daniela Micucci , Oliviero Riganelli

Large Language Models are transforming software engineering, yet prompt management in practice remains ad hoc, hindering reliability, reuse, and integration into industrial workflows. We present Prompt-with-Me, a practical solution for…

Software Engineering · Computer Science 2025-09-23 Ziyou Li , Agnia Sergeyuk , Maliheh Izadi

The use of large language model (LLM)-powered chatbots, such as ChatGPT, has become popular across various domains, supporting a range of tasks and processes. However, due to the intrinsic complexity of LLMs, effective prompting is more…

Advancements in Large Language Models (LLMs), such as ChatGPT, offer significant opportunities to enhance instructional support in introductory programming courses. While extensive research has explored the effectiveness of LLMs in…

Human-Computer Interaction · Computer Science 2025-05-09 Muntasir Hoq , Jessica Vandenberg , Shuyin Jiao , Seung Lee , Bradford Mott , Narges Norouzi , James Lester , Bita Akram

Instruction-tuned Language Models (ILMs) have become essential components of modern AI systems, demonstrating exceptional versatility across natural language and reasoning tasks. Among their most impactful applications is code generation,…

Software Engineering · Computer Science 2026-02-18 Zaiyu Cheng , Antonio Mastropaolo

Chat-based prompts respond with verbose linear-sequential texts, making it difficult to explore and refine ambiguous intents, back up and reinterpret, or shift directions in creative AI-assisted design work. AI-Instruments instead embody…

Human-Computer Interaction · Computer Science 2025-02-27 Nathalie Riche , Anna Offenwanger , Frederic Gmeiner , David Brown , Hugo Romat , Michel Pahud , Nicolai Marquardt , Kori Inkpen , Ken Hinckley