English
Related papers

Related papers: PARROT: A Benchmark for Evaluating LLMs in Cross-S…

200 papers

Text-to-SQL semantic parsing faces challenges in generalizing to cross-domain and complex queries. Recent research has employed a question decomposition strategy to enhance the parsing of complex SQL queries. However, this strategy…

Computation and Language · Computer Science 2023-10-23 Ben Eyal , Amir Bachar , Ophir Haroche , Moran Mahabi , Michael Elhadad

With Large Language Models' (LLMs) emergent abilities on code generation tasks, Text-to-SQL has become one of the most popular downstream applications. Despite the strong results of multiple recent LLM-based Text-to-SQL frameworks, the…

Machine Learning · Computer Science 2025-09-09 Dazhi Peng

In-context learning (ICL) has emerged as a new approach to various natural language processing tasks, utilizing large language models (LLMs) to make predictions based on context that has been supplemented with a few examples or…

Computation and Language · Computer Science 2023-05-23 Linyong Nan , Yilun Zhao , Weijin Zou , Narutatsu Ri , Jaesung Tae , Ellen Zhang , Arman Cohan , Dragomir Radev

As large language models (LLMs) continue to advance, the need for up-to-date and well-organized benchmarks becomes increasingly critical. However, many existing datasets are scattered, difficult to manage, and make it challenging to perform…

Machine Learning · Computer Science 2025-06-03 Eunsu Kim , Haneul Yoo , Guijin Son , Hitesh Patel , Amit Agarwal , Alice Oh

Despite recent progress in text-to-SQL parsing, current semantic parsers are still not accurate enough for practical use. In this paper, we investigate how to build automatic text-to-SQL error correction models. Noticing that token-level…

Computation and Language · Computer Science 2023-05-30 Ziru Chen , Shijie Chen , Michael White , Raymond Mooney , Ali Payani , Jayanth Srinivasa , Yu Su , Huan Sun

Text-to-SQL demands precise reasoning to convert natural language questions into structured queries. While large language models (LLMs) excel in many reasoning tasks, their ability to leverage Chain-of-Thought (CoT) reasoning for…

Machine Learning · Computer Science 2025-03-27 Bohan Zhai , Canwen Xu , Yuxiong He , Zhewei Yao

Large Language Models (LLMs) have demonstrated remarkable progress in translating natural language to SQL, but a significant semantic gap persists between their general knowledge and domain-specific semantics of databases. Historical…

Databases · Computer Science 2025-11-05 Yiwen Jiao , Tonghui Ren , Yuche Gao , Zhenying He , Yinan Jing , Kai Zhang , X. Sean Wang

The rise of large language models (LLMs) has enabled LLM-based applications (a.k.a. AI agents or co-pilots), a new software paradigm that combines the strength of LLM and conventional software. Diverse LLM applications from different…

Machine Learning · Computer Science 2024-05-31 Chaofan Lin , Zhenhua Han , Chengruidong Zhang , Yuqing Yang , Fan Yang , Chen Chen , Lili Qiu

Structured Query Language (SQL) has remained the standard query language for databases. SQL is highly optimized for processing structured data laid out in relations. Meanwhile, in the present application development landscape, it is highly…

Databases · Computer Science 2026-04-24 Udesh Kumarasinghe , Tyler Liu , Ahmed R. Mahmood , Chunwei Liu , Walid G. Aref

In-context learning with large language models (LLMs) has recently caught increasing attention due to its superior few-shot performance on various tasks. However, its performance on text-to-SQL parsing still has much room for improvement.…

Computation and Language · Computer Science 2023-10-30 Chang-You Tai , Ziru Chen , Tianshu Zhang , Xiang Deng , Huan Sun

Existing refinement methods in LLM-based Text-to-SQL systems exhibit limited effectiveness. They often introduce new errors during the self-correction process and fail to detect and correct semantic inaccuracies. To address these gaps, we…

Artificial Intelligence · Computer Science 2025-05-22 Jikai Chen , Leilei Gan , Ziyu Zhao , Zechuan Wang , Dong Wang , Chenyi Zhuang

The task of Text-to-SQL enables anyone to retrieve information from SQL databases using natural language. While this task has made substantial progress, the two primary evaluation metrics - Execution Accuracy (EXE) and Exact Set Matching…

Computation and Language · Computer Science 2025-06-18 Benjamin G. Ascoli , Yasoda Sai Ram Kandikonda , Jinho D. Choi

With the development of the Large Language Models (LLMs), a large range of LLM-based Text-to-SQL(Text2SQL) methods have emerged. This survey provides a comprehensive review of LLM-based Text2SQL studies. We first enumerate classic…

Computation and Language · Computer Science 2025-06-04 Liang Shi , Zhengju Tang , Nan Zhang , Xiaotong Zhang , Zhi Yang

Translating culture-related content is vital for effective cross-cultural communication. However, many culture-specific items (CSIs) often lack viable translations across languages, making it challenging to collect high-quality, diverse…

Computation and Language · Computer Science 2024-10-22 Binwei Yao , Ming Jiang , Tara Bobinac , Diyi Yang , Junjie Hu

Text-to-SQL is a subtask in semantic parsing that has seen rapid progress with the evolution of Large Language Models (LLMs). However, LLMs face challenges due to hallucination issues and a lack of domain-specific database knowledge(such as…

Computation and Language · Computer Science 2025-02-26 Xingyu Ma , Xin Tian , Lingxiang Wu , Xuepeng Wang , Xueming Tang , Jinqiao Wang

One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider…

Recent divide-and-conquer reasoning approaches, particularly those based on Chain-of-Thought (CoT), have substantially improved the Text-to-SQL capabilities of Large Language Models (LLMs). However, when applied to complex enterprise…

Computation and Language · Computer Science 2025-11-27 Zhifeng Hao , Qibin Song , Ruichu Cai , Boyan Xu

The construction of high-quality parallel corpora for translation research has increasingly evolved from simple sentence alignment to complex, multi-layered annotation tasks. This methodological shift presents significant challenges for…

Computation and Language · Computer Science 2026-02-12 Baorong Huang , Ali Asiri

Exploring the generalization of a text-to-SQL parser is essential for a system to automatically adapt the real-world databases. Previous works provided investigations focusing on lexical diversity, including the influence of the synonym and…

Artificial Intelligence · Computer Science 2023-01-24 Jieyu Li , Lu Chen , Ruisheng Cao , Su Zhu , Hongshen Xu , Zhi Chen , Hanchong Zhang , Kai Yu

With the future striving toward data-centric decision-making, seamless access to databases is of utmost importance. There is extensive research on creating an efficient text-to-sql (TEXT2SQL) model to access data from the database. Using a…

Computation and Language · Computer Science 2022-08-10 Ayush Kumar , Parth Nagarkar , Prabhav Nalhe , Sanjeev Vijayakumar