English
Related papers

Related papers: ScienceBenchmark: A Complex Real-World Benchmark f…

200 papers

There are many recent advanced developments for the Text-to-SQL task, where the Picard model is one of the the top performing models as measured by the Spider dataset competition. However, bringing Text-to-SQL systems to realistic use-cases…

Computation and Language · Computer Science 2023-12-12 Irene Manotas , Octavian Popescu , Ngoc Phuoc An Vo , Vadim Sheinin

Large language models (LLMs) have achieved strong performance on natural language to SQL (NL2SQL) benchmarks, yet their reported accuracy may be inflated by contamination from benchmark queries or structurally similar patterns seen during…

Computation and Language · Computer Science 2026-04-21 Mohammadtaher Safarzadeh , Hitesh Laxmichand Patel , Afshin Orojlooyjadid , Graham Horwood , Dan Roth

Text-to-SQL prompt strategies based on Large Language Models (LLMs) achieve remarkable performance on well-known benchmarks. However, when applied to real-world databases, their performance is significantly less than for these benchmarks,…

Large language models (LLMS) have shown increasing effectiveness in Text-to-SQL tasks. However, another closely related problem, Cross-System SQL Translation (a.k.a., SQL-to-SQL), which adapts a query written for one database system (e.g.,…

Databases · Computer Science 2025-09-30 Wei Zhou , Guoliang Li , Haoyu Wang , Yuxing Han , Xufei Wu , Fan Wu , Xuanhe Zhou

Machine translation is going through a radical revolution, driven by the explosive development of deep learning techniques using Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). In this paper, we consider a special…

Computation and Language · Computer Science 2018-06-12 Ruichu Cai , Boyan Xu , Xiaoyan Yang , Zhenjie Zhang , Zijian Li , Zhihao Liang

In order to reduce the manual cost of designing ER models, recent approaches have been proposed to address the task of NL2ERM, i.e., automatically generating entity-relationship (ER) models from natural language (NL) utterances such as…

Computation and Language · Computer Science 2023-12-22 Zhenwen Li , Jian-Guang Lou , Tao Xie

In addressing the pivotal role of translating natural language queries into SQL commands, we propose a suite of compact, fine-tuned models and self-refine mechanisms to democratize data access and analysis for non-expert users, mitigating…

Artificial Intelligence · Computer Science 2024-09-25 Lixia Wu , Peng Li , Junhong Lou , Lei Fu

This study presents AraSpider, the first Arabic version of the Spider dataset, aimed at improving natural language processing (NLP) in the Arabic-speaking community. Four multilingual translation models were tested for their effectiveness…

Computation and Language · Computer Science 2024-02-13 Ahmed Heakl , Youssef Mohamed , Ahmed B. Zaky

Existing text-to-SQL benchmarks have largely been constructed from public databases with well-structured schemas and simplistic question-SQL pairs. While large language models (LLMs) excel on these settings, their efficacy in complex…

Computation and Language · Computer Science 2026-05-14 Peter Baile Chen , Devin Yang , Weiyue Li , Fabian Wenz , Yi Zhang , Nesime Tatbul , Michael Cafarella , Çağatay Demiralp , Michael Stonebraker

Text-to-SQL enables users to interact with databases using natural language, simplifying the retrieval and synthesis of information. Despite the remarkable success of large language models (LLMs) in translating natural language questions…

Artificial Intelligence · Computer Science 2024-07-03 Gyubok Lee , Woosog Chay , Seonhee Cho , Edward Choi

Text-to-SQL systems have become crucial for translating natural language into SQL queries in various industries, enabling non-technical users to perform complex data operations. The need for accurate evaluation methods has increased as…

Computation and Language · Computer Science 2024-10-29 Heegyu Kim , Taeyang Jeon , Seunghwan Choi , Seungtaek Choi , Hyunsouk Cho

NoSQL databases have become increasingly popular due to their outstanding performance in handling large-scale, unstructured, and semi-structured data, highlighting the need for user-friendly interfaces to bridge the gap between…

Databases · Computer Science 2025-02-19 Jinwei Lu , Yuanfeng Song , Zhiqian Qin , Haodi Zhang , Chen Zhang , Raymond Chi-Wing Wong

The fundamental goal of the Text-to-SQL task is to translate natural language question into SQL query. Current research primarily emphasizes the information coupling between natural language questions and schemas, and significant progress…

Computation and Language · Computer Science 2024-01-01 Jiawen Yi , Guo Chen

To translate natural language questions into executable database queries, most approaches rely on a fully annotated training set. Annotating a large dataset with queries is difficult as it requires query-language expertise. We reduce this…

Computation and Language · Computer Science 2022-06-01 Irina Saparina , Anton Osokin

Natural language is hypothetically the best user interface for many domains. However, general models that provide an interface between natural language and any other domain still do not exist. Providing natural language interface to…

Computation and Language · Computer Science 2020-05-18 Jovan Kalajdjieski , Martina Toshevska , Frosina Stojanovska

Generating accurate SQL from users' natural language questions (text-to-SQL) remains a long-standing challenge due to the complexities involved in user question understanding, database schema comprehension, and SQL generation. Traditional…

Computation and Language · Computer Science 2025-11-25 Zijin Hong , Zheng Yuan , Qinggang Zhang , Hao Chen , Junnan Dong , Feiran Huang , Xiao Huang

In recent years,Text-to-SQL, the problem of automatically converting questions posed in natural language to formal SQL queries, has emerged as an important problem at the intersection of natural language processing and data management…

Computation and Language · Computer Science 2024-09-17 Ke Shen , Mayank Kejriwal

Text-to-SQL has emerged as a prominent research area, particularly with the rapid advancement of large language models (LLMs). By enabling users to query databases through natural language rather than SQL, this technology significantly…

Artificial Intelligence · Computer Science 2026-05-11 Yu-Jie Yang , Hung-Fu Chang , Po-An Chen

Most existing studies in text-to-SQL tasks do not require generating complex SQL queries with multiple clauses or sub-queries, and generalizing to new, unseen databases. In this paper we propose SyntaxSQLNet, a syntax tree network to…

Computation and Language · Computer Science 2018-10-29 Tao Yu , Michihiro Yasunaga , Kai Yang , Rui Zhang , Dongxu Wang , Zifan Li , Dragomir Radev

Text-to-SQL parsing is an essential and challenging task. The goal of text-to-SQL parsing is to convert a natural language (NL) question to its corresponding structured query language (SQL) based on the evidences provided by relational…

Computation and Language · Computer Science 2022-08-30 Bowen Qin , Binyuan Hui , Lihan Wang , Min Yang , Jinyang Li , Binhua Li , Ruiying Geng , Rongyu Cao , Jian Sun , Luo Si , Fei Huang , Yongbin Li