English
Related papers

Related papers: BlackBox Toolkit: Intelligent Assistance to UI Des…

200 papers

Studies of Generative AI (GenAI)-assisted creative workflows have focused on individuals overcoming challenges of prompting to produce what they envisioned. When designers work in teams, how do collaboration and prompting influence each…

Human-Computer Interaction · Computer Science 2025-09-29 Yuanning Han , Ziyi Qiu , Jiale Cheng , RAY LC

Artificial neural networks have proven to be extremely useful models that have allowed for multiple recent breakthroughs in the field of Artificial Intelligence and many others. However, they are typically regarded as black boxes, given how…

Artificial Intelligence · Computer Science 2023-03-07 Manuel de Sousa Ribeiro , João Leite

AI image models are rapidly evolving, disrupting aesthetic production in many industries. However, understanding of their underlying archives, their logic of image reproduction, and their persistent biases remains limited. What kind of…

Computers and Society · Computer Science 2023-07-20 Luke Munn , Liam Magee , Vanicka Arora

With recent advancements in the capabilities of Text-to-Image (T2I) AI models, product designers have begun experimenting with them in their work. However, T2I models struggle to interpret abstract language and the current user experience…

Human-Computer Interaction · Computer Science 2025-01-31 David Chuan-En Lin , Hyeonsu B. Kang , Nikolas Martelaro , Aniket Kittur , Yan-Ying Chen , Matthew K. Hong

This paper provides an overview of prominent deep learning toolkits and, in particular, reports on recent publications that contributed open source software for implementing tasks that are common in intelligent user interfaces (IUI). We…

Human-Computer Interaction · Computer Science 2018-03-15 Jan Zacharias , Michael Barz , Daniel Sonntag

Computer-Use Agents (CUA) are becoming increasingly capable of autonomously operating digital environments through Graphical User Interfaces (GUI). Yet, most GUI remain designed primarily for humans--prioritizing aesthetics and…

Computer Vision and Pattern Recognition · Computer Science 2025-11-20 Kevin Qinghong Lin , Siyuan Hu , Linjie Li , Zhengyuan Yang , Lijuan Wang , Philip Torr , Mike Zheng Shou

Artificial Intelligence (AI) planning is a flourishing research and development discipline that provides powerful tools for searching a course of action that achieves some user goal. While these planning tools show excellent performance on…

Artificial Intelligence · Computer Science 2021-02-23 Sebastian Graef , Ilche Georgievski

Many recent AI-powered UX design tools focus on generating individual static UI screens from natural language. However, they overlook the crucial aspect of interactions and user experiences across multiple screens. Through formative studies…

Human-Computer Interaction · Computer Science 2024-06-25 Yuwen Lu , Ziang Tong , Qinyi Zhao , Yewon Oh , Bryan Wang , Toby Jia-Jun Li

The practice of programming is undergoing a revolution with the introduction of AI assisted development (copilots) and the creation of new programming languages that are designed explicitly for tooling, analysis, and automation. Integrated…

Software Engineering · Computer Science 2024-06-17 Mark Marron

Customization has long been a central goal in interactive systems, yet prior work shows that end-user tailoring occurs infrequently and is often confined to initial setup or moments of breakdown. Recent advances in generative AI suggest…

Human-Computer Interaction · Computer Science 2026-05-13 Karthik Sreedhar , Aryan Kaul , Lydia B. Chilton

Conversational AI systems increasingly function as primary interfaces for information seeking, yet how they present sources to support information evaluation remains under-explored. This paper investigates how source transparency design…

Human-Computer Interaction · Computer Science 2026-01-22 Jiangen He , Jiqun Liu

Artificial Intelligence (AI) is one of the disruptive technologies that is shaping the future. It has growing applications for data-driven decisions in major smart city solutions, including transportation, education, healthcare, public…

Machine Learning · Computer Science 2021-11-02 M. Humayn Kabir , Khondokar Fida Hasan , Mohammad Kamrul Hasan , Keyvan Ansari

Human computer interaction is shifting from screen-based systems to multimodal interfaces where artificial intelligence powered systems increasingly interpret user intent through speech, gesture, and gaze. Yet users rarely understand how…

Human-Computer Interaction · Computer Science 2026-05-05 Ankur Bhatt , Sven Mayer

Since Artificial Intelligence (AI) software uses techniques like deep lookahead search and stochastic optimization of huge neural networks to fit mammoth datasets, it often results in complex behavior that is difficult for people to…

Artificial Intelligence · Computer Science 2018-10-16 Daniel S. Weld , Gagan Bansal

Machine generation of symbolic music and digital audio are hot topics but there have been relatively few digital musical instruments that integrate generative AI. Present musical AI tools are not artist centred and do not support…

Sound · Computer Science 2026-04-28 Charles Patrick Martin

We focus on the problem of designing an artificial agent (AI), capable of assisting a human user to complete a task. Our goal is to guide human users towards optimal task performance while keeping their cognitive load as low as possible.…

Robotics · Computer Science 2019-11-05 Gilwoo Lee , Christoforos Mavrogiannis , Siddhartha S. Srinivasa

Artificial intelligence (AI) and machine learning (ML) are increasingly integrated into the functioning of physical and digital products, creating unprecedented opportunities for interaction and functionality. However, there is a challenge…

Human-Computer Interaction · Computer Science 2023-10-03 Dave Murray-Rust , Maria Luce Lupetti , Iohanna Nicenboim , Wouter van der Hoog

Artificial Intelligence (AI) holds promise as a technology that can be used to improve government and economic policy-making. This paper proposes a new research agenda towards this end by introducing Social Environment Design, a general…

Artificial Intelligence · Computer Science 2024-06-18 Edwin Zhang , Sadie Zhao , Tonghan Wang , Safwan Hossain , Henry Gasztowtt , Stephan Zheng , David C. Parkes , Milind Tambe , Yiling Chen

A key objective in artificial intelligence (AI) development is to create systems that match or surpass human creativity. Although current AI models perform well across diverse creative tasks, it remains unclear whether these achievements…

Human-Computer Interaction · Computer Science 2025-04-01 Man Zhang , Ying Li , Yang Peng , Yijia Sun , Wenxin Guo , Huiqing Hu , Shi Chen , Qingbai Zhao

As interfaces evolve from static user pathways to dynamic human-AI collaboration, no standard methods exist for selecting appropriate interface patterns based on user needs and task complexity. Existing frameworks only provide guiding…

Human-Computer Interaction · Computer Science 2026-02-27 Shruthi Andru , Shrut Kirti Saksena