English
Related papers

Related papers: A Systematic Study of Performance Disparities in M…

200 papers

Language models based on the Transformer architecture have achieved state-of-the-art performance on a wide range of NLP tasks such as text classification, question-answering, and token classification. However, this performance is usually…

Computation and Language · Computer Science 2020-11-05 Kushal Jain , Adwait Deshpande , Kumar Shridhar , Felix Laumann , Ayushman Dash

Results reported in large-scale multilingual evaluations are often fragmented and confounded by factors such as target languages, differences in experimental setups, and model choices. We propose a framework that disentangles these…

Computation and Language · Computer Science 2025-08-26 Songbo Hu , Ivan Vulić , Anna Korhonen

Task-oriented dialogue (TOD) systems have assisted users on many tasks, including ticket booking and service inquiries. While existing TOD systems have shown promising performance in serving customer needs, these systems mostly assume that…

Computation and Language · Computer Science 2022-10-25 Zhiqiang Hu , Roy Kaa-Wei Lee , Nancy F. Chen

Creating multilingual task-oriented dialogue (TOD) agents is challenging due to the high cost of training data acquisition. Following the research trend of improving training data efficiency, we show for the first time, that in-context…

In this work, we provide a systematic and comprehensive empirical comparison of pretrained multilingual language models versus their monolingual counterparts with regard to their monolingual task performance. We study a set of nine…

Computation and Language · Computer Science 2021-06-03 Phillip Rust , Jonas Pfeiffer , Ivan Vulić , Sebastian Ruder , Iryna Gurevych

In Task-Oriented Dialogue (TOD) systems, correctly updating the system's understanding of the user's requests (\textit{a.k.a} dialogue state tracking) is key to a smooth interaction. Traditionally, TOD systems perform this update in three…

Computation and Language · Computer Science 2024-07-02 Lucas Druart , Valentin Vielzeuf , Yannick Estève

Pre-trained conversation models (PCMs) have achieved promising progress in recent years. However, existing PCMs for Task-oriented dialog (TOD) are insufficient for capturing the sequential nature of the TOD-related tasks, as well as for…

Computation and Language · Computer Science 2023-10-03 Lucen Zhong , Hengtong Lu , Caixia Yuan , Xiaojie Wang , Jiashen Sun , Ke Zeng , Guanglu Wan

Task-oriented dialogue (TOD) systems are experiencing a revolution driven by Large Language Models (LLMs), yet the evaluation methodologies for these systems remain insufficient for their growing sophistication. While traditional automatic…

Computation and Language · Computer Science 2025-07-17 Emre Can Acikgoz , Carl Guo , Suvodip Dey , Akul Datta , Takyoung Kim , Gokhan Tur , Dilek Hakkani-Tür

Task-oriented dialogue (TOD) systems are mainly based on the slot-filling-based TOD (SF-TOD) framework, in which dialogues are broken down into smaller, controllable units (i.e., slots) to fulfill a specific task. A series of approaches…

Large language models (LLMs) have demonstrated multilingual capabilities, yet they are mostly English-centric due to the imbalanced training corpora. While prior works have leveraged this bias to enhance multilingual performance through…

Computation and Language · Computer Science 2025-04-22 Chaoqun Liu , Wenxuan Zhang , Yiran Zhao , Anh Tuan Luu , Lidong Bing

Task-Oriented Dialogue (TOD) systems assist users in completing tasks through natural language interactions, often relying on a single-layered workflow structure for slot-filling in public tasks, such as hotel bookings. However, in…

Computation and Language · Computer Science 2025-09-09 Lingbo Mo , Shun Jiang , Akash Maharaj , Bernard Hishamunda , Yunyao Li

Task oriented dialogue (TOD) requires the complex interleaving of a number of individually controllable components with strong guarantees for explainability and verifiability. This has made it difficult to adopt the multi-turn multi-domain…

Computation and Language · Computer Science 2020-10-07 Oluwatobi O. Olabiyi , Prarthana Bhattarai , C. Bayan Bruss , Zachary Kulis

While the English virtual assistants have achieved exciting performance with an enormous amount of training resources, the needs of non-English-speakers have not been satisfied well. Up to Dec 2021, Alexa, one of the most popular smart…

Computation and Language · Computer Science 2021-12-24 Michael Sun , Kaili Huang , Mehrad Moradshahi

Cross-lingual transfer is a popular approach to increase the amount of training data for NLP tasks in a low-resource context. However, the best strategy to decide which cross-lingual data to include is unclear. Prior research often focuses…

Computation and Language · Computer Science 2025-05-22 Verena Blaschke , Masha Fedzechkina , Maartje ter Hoeve

End-to-end Task-Oriented Dialog (TOD) systems typically require extensive training datasets to perform well. In contrast, large language model (LLM) based TOD systems can excel even with limited data due to their ability to learn tasks…

Computation and Language · Computer Science 2024-10-21 Vishal Vivek Saley , Rocktim Jyoti Das , Dinesh Raghu , Mausam

The NLP research community has devoted increased attention to languages beyond English, resulting in considerable improvements for multilingual NLP. However, these improvements only apply to a small subset of the world's languages. Aiming…

Computation and Language · Computer Science 2024-10-03 Esther Ploeger , Wessel Poelman , Miryam de Lhoneux , Johannes Bjerva

Dialogue systems (DS), including the task-oriented dialogue system (TOD) and the open-domain dialogue system (ODD), have always been a fundamental task in natural language processing (NLP), allowing various applications in practice. Owing…

Computation and Language · Computer Science 2025-07-22 Hongru Wang , Lingzhi Wang , Yiming Du , Liang Chen , Jingyan Zhou , Yufei Wang , Kam-Fai Wong

This paper presents an ontology-aware pretrained language model (OPAL) for end-to-end task-oriented dialogue (TOD). Unlike chit-chat dialogue models, task-oriented dialogue models fulfill at least two task-specific modules: dialogue state…

Computation and Language · Computer Science 2022-09-13 Zhi Chen , Yuncong Liu , Lu Chen , Su Zhu , Mengyue Wu , Kai Yu

Due to the significance and value in human-computer interaction and natural language processing, task-oriented dialog systems are attracting more and more attention in both academic and industrial communities. In this paper, we survey…

Computation and Language · Computer Science 2020-06-24 Zheng Zhang , Ryuichi Takanobu , Qi Zhu , Minlie Huang , Xiaoyan Zhu

As a recent development, task-oriented dialogues (TODs) have been enriched with chitchat in an effort to make dialogues more diverse and engaging. This enhancement is particularly valuable as TODs are often confined to narrow domains,…

Computation and Language · Computer Science 2024-01-26 Armand Stricker , Patrick Paroubek