English
Related papers

Related papers: ChatVis: Large Language Model Agent for Generating…

200 papers

Large Language Models (LLMs) recently demonstrated capabilities for generating source code in common programming languages. Additionally, commercial products such as ChatGPT 4 started to provide code interpreters, allowing for the automatic…

Software Engineering · Computer Science 2023-11-15 Felix Härer

Translating natural language to visualization (NL2VIS) has shown great promise for visual data analysis, but it remains a challenging task that requires multiple low-level implementations, such as natural language processing and…

Human-Computer Interaction · Computer Science 2024-08-08 Nan Chen , Yuge Zhang , Jiahang Xu , Kan Ren , Yuqing Yang

We present a method for augmenting a Large Language Model (LLM) with a combination of text and visual data to enable accurate question answering in visualization of scientific data, making conversational visualization possible. LLMs…

Human-Computer Interaction · Computer Science 2025-01-17 Omar Mena , Alexandre Kouyoumdjian , Lonni Besançon , Michael Gleicher , Ivan Viola , Anders Ynnerman

Generative Large Language Models (LLMs) show potential in data analysis, yet their full capabilities remain uncharted. Our work explores the capabilities of LLMs for creating and refining visualizations via conversational interfaces. We…

Human-Computer Interaction · Computer Science 2023-11-10 Matt-Heun Hong , Anamaria Crisan

Large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks. Understanding how LLMs internally represent knowledge remains a significant challenge. Despite Sparse Autoencoders (SAEs) have…

Computation and Language · Computer Science 2025-09-26 Haoxuan Li , Zhen Wen , Qiqi Jiang , Chenxiao Li , Yuwei Wu , Yuchen Yang , Yiyao Wang , Xiuqi Huang , Minfeng Zhu , Wei Chen

Large Language Models(LLMs)have become effective tools for natural language processing and have been used in many different fields. This essay offers a succinct summary of various LLM subcategories. The survey emphasizes recent developments…

Computation and Language · Computer Science 2023-07-21 Saurabh Pahune , Manoj Chandrasekharan

Virtual Assistants (VAs) are important Information Retrieval platforms that help users accomplish various tasks through spoken commands. The speech recognition system (speech-to-text) uses query priors, trained solely on text, to…

Information Retrieval · Computer Science 2024-06-12 Sonal Sannigrahi , Thiago Fraga-Silva , Youssef Oualil , Christophe Van Gysel

Recent advances in large language models (LLMs) have enabled agentic systems that translate natural language intent into executable scientific visualization (SciVis) tasks. Despite rapid progress, the community lacks a principled and…

Large language models (LLMs) have demonstrated notable proficiency in code generation, with numerous prior studies showing their promising capabilities in various development scenarios. However, these studies mainly provide evaluations in…

Software Engineering · Computer Science 2024-03-19 Kailun Jin , Chung-Yu Wang , Hung Viet Pham , Hadi Hemmati

Large Language Models (LLMs) are being explored for applications in scientific research, including their capabilities to synthesize literature, answer research questions, generate research ideas, and even conduct computational experiments.…

Visualization is central to scientific discovery, yet authoring tools remain split between information and scientific visualization, and expertise in one rarely transfers to the other. Large Language Model (LLM) based systems promise to…

Human-Computer Interaction · Computer Science 2026-04-14 Alexandra Irger , Ella Hugie , Minghao Guo , Simon Warchol , Kenneth Moreland , David Pugmire , Wojciech Matusik , Hanspeter Pfister

Large Language Models (LLMs) have gained widespread popularity due to their ability to perform ad-hoc Natural Language Processing (NLP) tasks with a simple natural language prompt. Part of the appeal for LLMs is their approachability to the…

Human-Computer Interaction · Computer Science 2025-02-25 Aditi Mishra , Utkarsh Soni , Anjana Arunkumar , Jinbin Huang , Bum Chul Kwon , Chris Bryan

Large Language Models (LLMs) have emerged as coding assistants, capable of generating source code from natural language prompts. With the increasing adoption of LLMs in software development, academic research and industry based projects are…

Generating accurate and consistent visual aids is a critical challenge in mathematics education, where visual representations like geometric shapes and functions play a pivotal role in enhancing student comprehension. This paper introduces…

Computation and Language · Computer Science 2024-11-11 Jeongwoo Lee , Kwangsuk Park , Jihyeon Park

Data visualization generation using Large Language Models (LLMs) has shown promising results but often produces suboptimal visualizations that require human intervention for improvement. In this work, we introduce VIS-Shepherd, a…

Computer Vision and Pattern Recognition · Computer Science 2025-06-17 Bo Pan , Yixiao Fu , Ke Wang , Junyu Lu , Lunke Pan , Ziyang Qian , Yuhan Chen , Guoliang Wang , Yitao Zhou , Li Zheng , Yinghao Tang , Zhen Wen , Yuchen Wu , Junhua Lu , Biao Zhu , Minfeng Zhu , Bo Zhang , Wei Chen

Large language models (LLMs) have recently been applied in software engineering to perform tasks such as translating code between programming languages, generating code from natural language, and autocompleting code as it is being written.…

Human-Computer Interaction · Computer Science 2023-02-15 Steven I. Ross , Fernando Martinez , Stephanie Houde , Michael Muller , Justin D. Weisz

Recent advancements in large language models (LLMs) have demonstrated extraordinary comprehension capabilities with remarkable breakthroughs on various vision-language tasks. However, the application of LLMs in generating reliable medical…

Artificial Intelligence · Computer Science 2025-02-18 Xueshen Li , Xinlong Hou , Ziyi Huang , Yu Gan

We propose leveraging Large Language Models (LLMs) as an interaction layer for medical visualization systems. In domains like healthcare, where users must navigate high-dimensional, coded, and heterogeneous datasets, LLM-generated queries…

Human-Computer Interaction · Computer Science 2025-09-12 Ambre Assor , Hyeon Jeon , Sungbok Shin , Jean-Daniel Fekete

With the rise of multimodal applications, instruction data has become critical for training multimodal language models capable of understanding complex image-based queries. Existing practices rely on powerful but costly large language…

Computer Vision and Pattern Recognition · Computer Science 2024-12-31 Jieyu Zhang , Le Xue , Linxin Song , Jun Wang , Weikai Huang , Manli Shu , An Yan , Zixian Ma , Juan Carlos Niebles , Silvio Savarese , Caiming Xiong , Zeyuan Chen , Ranjay Krishna , Ran Xu

Recent advances in multi-modal large language models (MLLMs) have enabled increasingly sophisticated autonomous visualization agents capable of translating user intentions into data visualizations. However, measuring progress and comparing…

Human-Computer Interaction · Computer Science 2025-09-19 Kuangshi Ai , Haichao Miao , Zhimin Li , Chaoli Wang , Shusen Liu