English
Related papers

Related papers: An Exploratory Study of ML Sketches and Visual Cod…

200 papers

The quality of training data is critical to the performance of machine learning applications in domains like transportation, healthcare, and robotics. Accurate image labeling, however, often relies on time-consuming, expert-driven methods…

Human-Computer Interaction · Computer Science 2025-05-28 Baichuan Li , Larry Powell , Tracy Hammond

Large Language Models (LLMs) represent a leap in artificial intelligence, excelling in tasks using human language(s). Although the main focus of general-purpose LLMs is not code generation, they have shown promising results in the domain.…

Software Engineering · Computer Science 2024-01-30 Sanka Rasnayaka , Guanlin Wang , Ridwan Shariffdeen , Ganesh Neelakanta Iyer

We introduce the concept of code shaping, an interaction paradigm for editing code using free-form sketch annotations directly on top of the code and console output. To evaluate this concept, we conducted a three-stage design study with 18…

Human-Computer Interaction · Computer Science 2025-02-11 Ryan Yen , Jian Zhao , Daniel Vogel

Coding agents powered by large language models (LLMs) have gained traction for automating code generation through iterative problem-solving with minimal human involvement. Despite the emergence of various frameworks, e.g., LangChain,…

Machine Learning · Computer Science 2025-08-19 Junpeng Wang , Yuzhong Chen , Menghai Pan , Chin-Chia Michael Yeh , Mahashweta Das

Early-stage design ideation often relies on rough sketches created under time pressure, leaving much of the designer's intent implicit. In practice, designers frequently speak while sketching, verbally articulating functional goals and…

Human-Computer Interaction · Computer Science 2026-04-15 Weiyan Shi , Dorien Herremans , Kenny Tsu Wei Choo

Sketches have been used to conceptualise and depict visual objects from pre-historic times. Sketch research has flourished in the past decade, particularly with the proliferation of touchscreen devices. Much of the utilisation of sketch has…

Computer Vision and Pattern Recognition · Computer Science 2022-10-28 Ayan Kumar Bhunia

Machine learning practitioners often end up tunneling on low-level technical details like model architectures and performance metrics. Could early model development instead focus on high-level questions of which factors a model ought to pay…

Human-Computer Interaction · Computer Science 2023-03-07 Michelle S. Lam , Zixian Ma , Anne Li , Izequiel Freitas , Dakuo Wang , James A. Landay , Michael S. Bernstein

Sketching is an important activity for understanding, designing, and communicating different aspects of software systems such as their requirements or architecture. Often, sketches start on paper or whiteboards, are revised, and may evolve…

Software Engineering · Computer Science 2017-08-08 Sebastian Baltes , Fabrice Hollerich , Stephan Diehl

Large language model (LLM)-based coding assistants have made substantial progress, yet most systems remain reactive, requiring developers to explicitly formulate their needs. Proactive coding assistants aim to infer latent developer intent…

Software Engineering · Computer Science 2026-05-08 Lehui Li , Ruixuan Jia , Guo-Ye Yang , Jia Li

The evolution of LLM has resulted in coding-focused models that are able to produce code snippets with high accuracy. More and more AI coding assistant tools are now available, leading to greater integration of AI coding assistants into…

Software Engineering · Computer Science 2026-04-14 Neha Rani , Jeevan Ram Munnangi , Austin Matthew Spangler , Donald Honeycutt

Charts are high-density visualization carriers for complex data, serving as a crucial medium for information extraction and analysis. Automated chart understanding poses significant challenges to existing multimodal large language models…

Computer Vision and Pattern Recognition · Computer Science 2025-05-27 Muye Huang , Lingling Zhang , Jie Ma , Han Lai , Fangzhi Xu , Yifei Li , Wenjun Wu , Yaqiang Wu , Jun Liu

Drawing tools for Virtual Reality (VR) enable users to model 3D designs from within the virtual environment itself. These tools employ sketching and sculpting techniques known from desktop-based interfaces and apply them to hand-based…

Human-Computer Interaction · Computer Science 2019-11-06 Daniele Giunchi , Stuart james , Donald Degraen , Anthony Steed

Recent advances in large language models (LLMs) have introduced new paradigms in software development, including vibe coding, AI-assisted coding, and agentic coding, fundamentally reshaping how software is designed, implemented, and…

Software Engineering · Computer Science 2026-01-01 Hung-Fu Chang , MohammadShokrolah Shirazi , Lizhou Cao , Supannika Koolmanojwong Mobasser

For a full-stack web or app development, it requires a software firm or more specifically a team of experienced developers to contribute a large portion of their time and resources to design the website and then convert it to code. As a…

Computer Vision and Pattern Recognition · Computer Science 2022-11-29 Somoy Subandhu Barua , Imam Mohammad Zulkarnain , Abhishek Roy , Md. Golam Rabiul Alam , Md Zia Uddin

The practice of programming is undergoing a revolution with the introduction of AI assisted development (copilots) and the creation of new programming languages that are designed explicitly for tooling, analysis, and automation. Integrated…

Software Engineering · Computer Science 2024-06-17 Mark Marron

Coding forms a key part of computer science education in universities. As part of this education, Integrated Development Environments (IDEs) are essential tools for coding. However, it is currently unknown how the design of an IDE's…

Human-Computer Interaction · Computer Science 2025-06-13 Luke Halpin , Phillip Benachour , Tracy Hall , Ann-Marie Houghton , Emily Winter

The integration of AI assistants into software development workflows is rapidly evolving, shifting from automation-assisted tasks to collaborative interactions between developers and AI. Large Language Models (LLMs) have demonstrated their…

Software Engineering · Computer Science 2025-06-16 Benedetta Donato , Leonardo Mariani , Daniela Micucci , Oliviero Riganelli , Marco Somaschini

Code completion plays a prominent role in modern integrated development environments (IDEs). Machine learning has become ubiquitous in analogous natural language writing and search software, surfacing more relevant autocompletions and…

Software Engineering · Computer Science 2020-04-14 Gareth Ari Aye , Gail E. Kaiser

Learning data storytelling involves a complex web of skills. Professional and academic educational offerings typically focus on the computational literacies required, but professionals in the field employ many non-technical methods;…

Human-Computer Interaction · Computer Science 2021-08-24 R. Bhargava , D. Williams , D. D'Ignazio

With recent advances in graphical user interfaces, more and more tasks on computers have become easier to perform. Out of the belief that creating computer programs can also be one of them, visual programming languages (VPLs) have emerged.…

Programming Languages · Computer Science 2012-06-25 Drazen Lucanin , Ivan Fabek