English
Related papers

Related papers: Vision2Web: A Hierarchical Benchmark for Visual We…

200 papers

Vision-Language Models (VLMs) have demonstrated impressive capabilities in code generation across various domains. However, their ability to replicate complex, multi-panel visualizations from real-world data remains largely unassessed. To…

The webpage-to-code task requires models to understand visual representations of webpages and generate corresponding code. However, existing benchmarks primarily focus on static screenshot-to-code tasks, thereby overlooking the dynamic…

Automating the conversion of UI images into web code is a critical task for front-end development and rapid prototyping. Advances in multimodal large language models (MLLMs) have made WebUI-to-Code increasingly feasible, yet existing…

Artificial Intelligence · Computer Science 2025-10-10 Peichao Lai , Jinhui Zhuang , Kexuan Zhang , Ningchang Xiong , Shengjie Wang , Yanwei Xu , Chong Chen , Yilei Wang , Bin Cui

As digitalization and cloud technologies evolve, the web is becoming increasingly important in the modern society. Autonomous web agents based on large language models (LLMs) hold a great potential in work automation. It is therefore…

Artificial Intelligence · Computer Science 2025-10-09 Tianci Xue , Weijian Qi , Tianneng Shi , Chan Hee Song , Boyu Gou , Dawn Song , Huan Sun , Yu Su

The emergence of "vibe coding" platforms, where users describe applications in natural language and AI agents autonomously generate full-stack software, has created a need for rigorous evaluation beyond code-level benchmarks. In order to…

Multiagent Systems · Computer Science 2026-05-07 Siddhant Saxena , Nilesh Trivedi , Vinayaka Jyothi

User interface (UI) development requires translating design mockups into functional code, a process that remains repetitive and labor-intensive. While recent Vision-Language Models (VLMs) automate UI-to-Code generation, they generate only…

Software Engineering · Computer Science 2025-11-11 Mingde Xu , Zhen Yang , Wenyi Hong , Lihang Pan , Xinyue Fan , Yan Wang , Xiaotao Gu , Bin Xu , Jie Tang

Translating natural language to visualization (NL2VIS) has shown great promise for visual data analysis, but it remains a challenging task that requires multiple low-level implementations, such as natural language processing and…

Human-Computer Interaction · Computer Science 2024-08-08 Nan Chen , Yuge Zhang , Jiahang Xu , Kan Ren , Yuqing Yang

At present, executable visual workflows have emerged as a mainstream paradigm in real-world industrial deployments, offering strong reliability and controllability. However, in current practice, such workflows are almost entirely…

Computation and Language · Computer Science 2026-05-27 Yi Zhong , Buqiang Xu , Yijun Wang , Zifei Shan , Shuofei Qiao , Guozhou Zheng , Ningyu Zhang

The rapid advancement of Multimodal Large Language Models (MLLMs) has enabled browsing agents to acquire and reason over multimodal information in the real world. But existing benchmarks suffer from two limitations: insufficient evaluation…

Computer Vision and Pattern Recognition · Computer Science 2026-03-19 Zhengbo Zhang , Jinbo Su , Zhaowen Zhou , Changtao Miao , Yuhan Hong , Qimeng Wu , Yumeng Liu , Feier Wu , Yihe Tian , Yuhao Liang , Zitong Shan , Wanke Xia , Yi-Fan Zhang , Bo Zhang , Zhe Li , Shiming Xiang , Ying Yan

Large language models are rapidly evolving into interactive coding agents capable of end-to-end web coding, yet existing benchmarks evaluate only narrow slices of this capability, typically text-conditioned generation with…

Building Graphical User Interface (GUI) assistants holds significant promise for enhancing human workflow productivity. While most agents are language-based, relying on closed-source API with text-rich meta-information (e.g., HTML or…

Computer Vision and Pattern Recognition · Computer Science 2024-11-27 Kevin Qinghong Lin , Linjie Li , Difei Gao , Zhengyuan Yang , Shiwei Wu , Zechen Bai , Weixian Lei , Lijuan Wang , Mike Zheng Shou

Agents powered by large language models (LLMs) are increasingly adopted in the software industry, contributing code as collaborators or even autonomous developers. As their presence grows, it becomes important to assess the current…

Software Engineering · Computer Science 2026-02-12 Qixing Zhou , Jiacheng Zhang , Haiyang Wang , Rui Hao , Jiahe Wang , Minghao Han , Yuxue Yang , Shuzhe Wu , Feiyang Pan , Lue Fan , Dandan Tu , Zhaoxiang Zhang

We introduce AI2Apps, a Visual Integrated Development Environment (Visual IDE) with full-cycle capabilities that accelerates developers to build deployable LLM-based AI agent Applications. This Visual IDE prioritizes both the Integrity of…

Artificial Intelligence · Computer Science 2024-04-09 Xin Pang , Zhucong Li , Jiaxiang Chen , Yuan Cheng , Yinghui Xu , Yuan Qi

Video production workflows offer a rich and demanding arena for evaluating multimodal AI agents: they require composite capabilities across text, image, audio, and video understanding, along with long-horizon planning, and tool use. To this…

Cryptography and Security · Computer Science 2026-05-28 Zongheng Cao , Yi Zheng , Rui Song , Xinyu Hu

Agent systems powered by large language models (LLMs) have demonstrated impressive performance on repository-level code-generation tasks. However, for tasks such as website codebase generation, which depend heavily on visual effects and…

Computation and Language · Computer Science 2025-09-29 Zimu Lu , Houxing Ren , Yunqiao Yang , Ke Wang , Zhuofan Zong , Junting Pan , Mingjie Zhan , Hongsheng Li

Agentic search such as Deep Research systems-where agents autonomously browse the web, synthesize information, and return comprehensive citation-backed answers-represents a major shift in how users interact with web-scale information. While…

Large language model-based web agents have demonstrated strong performance on realistic web interaction tasks. However, existing evaluations are predominantly conducted under relatively stable and well-behaved interaction conditions, which…

Software Engineering · Computer Science 2026-04-21 Haoyue Bai , Dong Wang , Long Chen , Bingguang Hao , Pengyang Shao , Yonghui Yang , Yicheng He , Chenyi Zhuang

User interface (UI) design is an iterative process in which designers progressively refine their work with design software such as Figma or Sketch. Recent advances in vision language models (VLMs) with tool invocation suggest these models…

Computer Vision and Pattern Recognition · Computer Science 2025-12-01 Daeheon Jeong , Seoyeon Byun , Kihoon Son , Dae Hyun Kim , Juho Kim

Multimodal large language models (MLLMs) have shown impressive success across modalities such as image, video, and audio in a variety of understanding and generation tasks. However, current MLLMs are surprisingly poor at understanding…

We present VISTAR, a user-centric, multi-dimensional benchmark for text-to-image (T2I) evaluation that addresses the limitations of existing metrics. VISTAR introduces a two-tier hybrid paradigm: it employs deterministic, scriptable metrics…

Computer Vision and Pattern Recognition · Computer Science 2025-08-11 Kaiyuan Jiang , Ruoxi Sun , Ying Cao , Yuqi Xu , Xinran Zhang , Junyan Guo , ChengSheng Deng