English
Related papers

Related papers: Ditch the Gold Standard: Re-evaluating Conversatio…

200 papers

We often rely on our intuition to anticipate the direction of a conversation. Endowing automated systems with similar foresight can enable them to assist human-human interactions. Recent work on developing models with this predictive…

Computation and Language · Computer Science 2025-07-28 Son Quoc Tran , Tushaar Gangavarapu , Nicholas Chernogor , Jonathan P. Chang , Cristian Danescu-Niculescu-Mizil

\Ac{LFQA} aims to generate lengthy answers to complex questions. This scenario presents great flexibility as well as significant challenges for evaluation. Most evaluations rely on deterministic metrics that depend on string or n-gram…

Information Retrieval · Computer Science 2025-04-28 Ning Xian , Yixing Fan , Ruqing Zhang , Maarten de Rijke , Jiafeng Guo

The increasing reliance on digital information necessitates advancements in conversational search systems, particularly in terms of information transparency. While prior research in conversational information-seeking has concentrated on…

Information Retrieval · Computer Science 2024-05-07 Weronika Łajewska , Damiano Spina , Johanne Trippas , Krisztian Balog

Product review websites provide an incredible lens into the wide variety of opinions and experiences of different people, and play a critical role in helping users discover products that match their personal needs and preferences. To help…

Information Retrieval · Computer Science 2016-10-27 Mengting Wan , Julian McAuley

As one promising way to inquire about any particular information through a dialog with the bot, question answering dialog systems have gained increasing research interests recently. Designing interactive QA systems has always been a…

Computation and Language · Computer Science 2021-04-26 Munazza Zaib , Dai Hoang Tran , Subhash Sagar , Adnan Mahmood , Wei E. Zhang , Quan Z. Sheng

Question Answering (QA) is a challenging topic since it requires tackling the various difficulties of natural language understanding. Since evaluation is important not only for identifying the strong and weak points of the various…

Computation and Language · Computer Science 2021-02-09 Katerina Papantoniou , Yannis Tzitzikas

Conversational recommender systems aim to interactively support online users in their information search and decision-making processes in an intuitive way. With the latest advances in voice-controlled devices, natural language processing,…

Information Retrieval · Computer Science 2022-10-31 Dietmar Jannach

Traditional methods for eliciting people's opinions face a trade-off between depth and scale: structured surveys enable large-scale data collection but limit respondents' ability to voice their opinions in their own words, while…

Human-Computer Interaction · Computer Science 2025-03-13 Alexander Wuttke , Matthias Aßenmacher , Christopher Klamm , Max M. Lang , Quirin Würschinger , Frauke Kreuter

A question answering (QA) system is a type of conversational AI that generates natural language answers to questions posed by human users. QA systems often form the backbone of interactive dialogue systems, and have been studied extensively…

Software Engineering · Computer Science 2021-01-12 Aakash Bansal , Zachary Eberhart , Lingfei Wu , Collin McMillan

Conversational Question Answering is a challenging task since it requires understanding of conversational history. In this project, we propose a new system RoBERTa + AT +KD, which involves rationale tagging multi-task, adversarial training,…

Computation and Language · Computer Science 2019-09-25 Ying Ju , Fubang Zhao , Shijie Chen , Bowen Zheng , Xuefeng Yang , Yunfeng Liu

With a lot of work about context-free question answering systems, there is an emerging trend of conversational question answering models in the natural language processing field. Thanks to the recently collected datasets, including QuAC and…

Computation and Language · Computer Science 2019-11-28 Ting-Rui Chiang , Hao-Tong Ye , Yun-Nung Chen

While conversing with chatbots, humans typically tend to ask many questions, a significant portion of which can be answered by referring to large-scale knowledge graphs (KG). While Question Answering (QA) and dialog systems have been…

Computation and Language · Computer Science 2018-10-05 Amrita Saha , Vardaan Pahuja , Mitesh M. Khapra , Karthik Sankaranarayanan , Sarath Chandar

Conversational question answering (QA) requires the ability to correctly interpret a question in the context of previous conversation turns. We address the conversational QA task by decomposing it into question rewriting and question…

Information Retrieval · Computer Science 2020-10-26 Svitlana Vakulenko , Shayne Longpre , Zhucheng Tu , Raviteja Anantha

Question-answering (QA) that comes naturally to humans is a critical component in seamless human-computer interaction. It has emerged as one of the most convenient and natural methods to interact with the web and is especially desirable in…

Computation and Language · Computer Science 2022-11-15 Deepak Gupta

We introduce AVA, an automatic evaluation approach for Question Answering, which given a set of questions associated with Gold Standard answers, can estimate system Accuracy. AVA uses Transformer-based language models to encode question,…

Computation and Language · Computer Science 2021-08-17 Thuy Vu , Alessandro Moschitti

Conversational question--answer generation is a task that automatically generates a large-scale conversational question answering dataset based on input passages. In this paper, we introduce a novel framework that extracts question-worthy…

Computation and Language · Computer Science 2022-09-26 Seonjeong Hwang , Gary Geunbae Lee

Question answering (QA) is an important natural language processing (NLP) task and has received much attention in academic research and industry communities. Existing QA studies assume that questions are raised by humans and answers are…

Computation and Language · Computer Science 2019-01-15 Qing Yin , Guan Luo , Xiaodong Zhu , Qinghua Hu , Ou Wu

To evaluate Large Language Models (LLMs) for question answering (QA), traditional methods typically focus on assessing single-turn responses to given questions. However, this approach doesn't capture the dynamic nature of human-AI…

Computation and Language · Computer Science 2024-11-19 Ruosen Li , Ruochen Li , Barry Wang , Xinya Du

Conversational Question Answering (ConvQA) systems have emerged as a pivotal area within Natural Language Processing (NLP) by driving advancements that enable machines to engage in dynamic and context-aware conversations. These capabilities…

Computation and Language · Computer Science 2025-09-09 Manoj Madushanka Perera , Adnan Mahmood , Kasun Eranda Wijethilake , Fahmida Islam , Maryam Tahermazandarani , Quan Z. Sheng

Do question answering (QA) modeling improvements (e.g., choice of architecture and training procedure) hold consistently across the diverse landscape of QA benchmarks? To study this question, we introduce the notion of concurrence -- two…

Computation and Language · Computer Science 2023-06-01 Nelson F. Liu , Tony Lee , Robin Jia , Percy Liang