English
Related papers

Related papers: Spider 2.0: Evaluating Language Models on Real-Wor…

200 papers

Text-to-SQL generation enables non-experts to interact with databases via natural language. Recent advances rely on large closed-source models like GPT-4 that present challenges in accessibility, privacy, and latency. To address these…

Computation and Language · Computer Science 2025-02-18 Satya Krishna Gorti , Ilan Gofman , Zhaoyan Liu , Jiapeng Wu , Noël Vouitsis , Guangwei Yu , Jesse C. Cresswell , Rasa Hosseinzadeh

Large Language Models (LLMs) have demonstrated strong performance on various tasks. To unleash their power on the Text-to-SQL task, we propose $R^3$ (Review-Rebuttal-Revision), a consensus-based multi-agent system for Text-to-SQL tasks.…

Computation and Language · Computer Science 2024-07-12 Hanchen Xia , Feng Jiang , Naihao Deng , Cunxiang Wang , Guojiang Zhao , Rada Mihalcea , Yue Zhang

NL2SQL (natural language to SQL) translates natural language questions into SQL queries, thereby making structured data accessible to non-technical users, serving as the foundation for intelligent data applications. State-of-the-art NL2SQL…

Text-to-SQL is a fundamental yet challenging task in the NLP area, aiming at translating natural language questions into SQL queries. While recent advances in large language models have greatly improved performance, most existing approaches…

Databases · Computer Science 2025-12-01 Shuozhi Yuan , Limin Chen , Miaomiao Yuan , Zhao Jin

Text-to-SQL, the task of translating natural language questions into SQL queries, plays a crucial role in enabling non-experts to interact with databases. While recent advancements in large language models (LLMs) have significantly enhanced…

Computation and Language · Computer Science 2025-07-15 Haoyang Li , Shang Wu , Xiaokang Zhang , Xinmei Huang , Jing Zhang , Fuxin Jiang , Shuai Wang , Tieying Zhang , Jianjun Chen , Rui Shi , Hong Chen , Cuiping Li

This study presents AraSpider, the first Arabic version of the Spider dataset, aimed at improving natural language processing (NLP) in the Arabic-speaking community. Four multilingual translation models were tested for their effectiveness…

Computation and Language · Computer Science 2024-02-13 Ahmed Heakl , Youssef Mohamed , Ahmed B. Zaky

Large Language Models (LLMs) struggle with complex Text-to-SQL queries that demand both sophisticated mathematical reasoning and intricate schema navigation. Existing methods often tackle these challenges in isolation, creating a fractured…

Artificial Intelligence · Computer Science 2025-09-25 Xutao Mao , Tao Liu , Hongying Zan

Translating natural language queries into SQLs in a seq2seq manner has attracted much attention recently. However, compared with abstract-syntactic-tree-based SQL generation, seq2seq semantic parsers face much more challenges, including…

Computation and Language · Computer Science 2023-06-16 Yuntao Li , Zhenpeng Su , Yutian Li , Hanchu Zhang , Sirui Wang , Wei Wu , Yan Zhang

This paper presents the development process of a natural language to SQL model using the T5 model as the basis. The models, developed in August 2022 for an online transaction processing system and a data warehouse, have a 73\% and 84\%…

SQL is central to enterprise data engineering, yet generating fully correct SQL code in a single attempt remains difficult, even for experienced developers and advanced text-to-SQL LLMs, often requiring multiple debugging iterations. We…

Artificial Intelligence · Computer Science 2026-01-27 Jing Ye , Yiwen Duan , Yonghong Yu , Victor Ma , Yang Gao , Xing Chen

Natural Language to SQL (i.e., NL2SQL) translation is crucial for democratizing database access, but even state-of-the-art models frequently generate semantically incorrect SQL queries, hindering the widespread adoption of these techniques…

Databases · Computer Science 2025-12-08 Xinyu Liu , Shuyu Shen , Boyan Li , Nan Tang , Yuyu Luo

Using the best Text-to-SQL methods in resource-constrained environments is challenging due to their reliance on resource-intensive open-source models. This paper introduces Auto Prompt SQL(AP-SQL), a novel architecture designed to bridge…

Computation and Language · Computer Science 2025-09-03 Zetong Tang , Qian Ma , Di Wu

Advances in large language models have accelerated progress in text-to-SQL, methods for converting natural language queries into valid SQL queries. A key bottleneck for developing generalizable text-to-SQL models is the lack of large-scale…

Information Retrieval · Computer Science 2026-02-27 Cornelius Wolff , Daniel Gomm , Madelon Hulsebos

We present a neural approach called IRNet for complex and cross-domain Text-to-SQL. IRNet aims to address two challenges: 1) the mismatch between intents expressed in natural language (NL) and the implementation details in SQL; 2) the…

Computation and Language · Computer Science 2019-05-30 Jiaqi Guo , Zecheng Zhan , Yan Gao , Yan Xiao , Jian-Guang Lou , Ting Liu , Dongmei Zhang

Recent advances in large language models (LLMs) have significantly improved the accuracy of Text-to-SQL systems. However, a critical challenge remains: the semantic mismatch between natural language questions (NLQs) and their corresponding…

Computation and Language · Computer Science 2025-08-21 Shaoming Duan , Zirui Wang , Chuanyi Liu , Zhibin Zhu , Yuhao Zhang , Peiyi Han , Liang Yan , Zewu Peng

To be informative, an evaluation must measure how well systems generalize to realistic unseen data. We identify limitations of and propose improvements to current evaluations of text-to-SQL systems. First, we compare human-generated and…

Computation and Language · Computer Science 2020-06-05 Catherine Finegan-Dollak , Jonathan K. Kummerfeld , Li Zhang , Karthik Ramanathan , Sesh Sadasivam , Rui Zhang , Dragomir Radev

Although multi-agent collaborative Large Language Models (LLMs) have achieved significant breakthroughs in the Text-to-SQL task, their performance is still constrained by various factors. These factors include the incompleteness of the…

Computation and Language · Computer Science 2025-02-24 Xiangjin Xie , Guangwei Xu , Lingyan Zhao , Ruijie Guo

Recent advancements in Text-to-SQL have pushed database management systems towards greater democratization of data access. Today's language models are at the core of these advancements. They enable impressive Text-to-SQL generation as…

Computation and Language · Computer Science 2024-06-19 Karime Maamari , Amine Mhedhbi

Text-to-SQL systems have achieved strong performance on English benchmarks, yet their behavior in morphologically rich, low-resource languages remains largely unexplored. We introduce BIRDTurk, the first Turkish adaptation of the BIRD…

When translating natural language questions into SQL queries to answer questions from a database, we would like our methods to generalize to domains and database schemas outside of the training set. To handle complex questions and database…

Machine Learning · Computer Science 2019-06-28 Richard Shin