English
Related papers

Related papers: Ditch the Gold Standard: Re-evaluating Conversatio…

200 papers

Rather than using (proxies of) end user or expert judgment to decide on the ranking of information, this paper asks whether conversations about information quality might offer a feasible and valuable addition for ranking information. We…

Information Retrieval · Computer Science 2022-10-17 Frans van der Sluis

While XAI focuses on providing AI explanations to humans, can the reverse - humans explaining their judgments to AI - foster richer, synergistic human-AI systems? This paper explores various forms of human inputs to AI and examines how…

Computers and Society · Computer Science 2025-03-07 Alan Dix , Tommaso Turchi , Ben Wilson , Anna Monreale , Matt Roach

In spoken dialogue systems, we aim to deploy artificial intelligence to build automated dialogue agents that can converse with humans. Dialogue systems are increasingly being designed to move beyond just imitating conversation and also…

Computation and Language · Computer Science 2021-11-03 Atharv Singh Patlan , Shiven Tripathi , Shubham Korde

Conversational agents promise conversational interaction but fail to deliver. Efforts often emulate functional rules from human speech, without considering key characteristics that conversation must encapsulate. Given its potential in…

Existing open-domain dialogue generation models are usually trained to mimic the gold response in the training set using cross-entropy loss on the vocabulary. However, a good response does not need to resemble the gold response, since there…

Computation and Language · Computer Science 2020-10-06 Wei-Jen Ko , Avik Ray , Yilin Shen , Hongxia Jin

Intelligent personal assistant systems that are able to have multi-turn conversations with human users are becoming increasingly popular. Most previous research has been focused on using either retrieval-based or generation-based methods to…

Information Retrieval · Computer Science 2019-08-27 Liu Yang , Junjie Hu , Minghui Qiu , Chen Qu , Jianfeng Gao , W. Bruce Croft , Xiaodong Liu , Yelong Shen , Jingjing Liu

With the rise of human-machine communication, machines are increasingly designed with humanlike characteristics, such as gender, which can inadvertently trigger cognitive biases. Many conversational agents (CAs), such as voice assistants…

Human-Computer Interaction · Computer Science 2024-01-09 Weizi Liu

Question answering (QA) is an important aspect of open-domain conversational agents, garnering specific research focus in the conversational QA (ConvQA) subtask. One notable limitation of recent ConvQA efforts is the response being answer…

Computation and Language · Computer Science 2020-12-18 Ashutosh Baheti , Alan Ritter , Kevin Small

AI is promising in assisting UX evaluators with analyzing usability tests, but its judgments are typically presented as non-interactive visualizations. Evaluators may have questions about test recordings, but have no way of asking them.…

Human-Computer Interaction · Computer Science 2023-03-08 Emily Kuang , Ehsan Jahangirzadeh Soure , Mingming Fan , Jian Zhao , Kristen Shinohara

Models for conversational question answering (ConvQA) over knowledge graphs (KGs) are usually trained and tested on benchmarks of gold QA pairs. This implies that training is limited to surface forms seen in the respective datasets, and…

Computation and Language · Computer Science 2024-02-20 Magdalena Kaiser , Rishiraj Saha Roy , Gerhard Weikum

The ability to engage in goal-oriented conversations has allowed humans to gain knowledge, reduce uncertainty, and perform tasks more efficiently. Artificial agents, however, are still far behind humans in having goal-driven conversations.…

Computation and Language · Computer Science 2019-07-30 Pushkar Shukla , Carlos Elmadjian , Richika Sharan , Vivek Kulkarni , Matthew Turk , William Yang Wang

Conversational search presents opportunities to support users in their search activities to improve the effectiveness and efficiency of search while reducing their cognitive load. Limitations of the potential competency of conversational…

Human-Computer Interaction · Computer Science 2021-04-12 Abhishek Kaushik , Gareth J. F. Jones

Community question answering (CQA) represents the type of Web applications where people can exchange knowledge via asking and answering questions. One significant challenge of most real-world CQA systems is the lack of effective matching…

Social and Information Networks · Computer Science 2018-07-18 Xianzhi Wang , Chaoran Huang , Lina Yao , Boualem Benatallah , Manqing Dong

The ability of a machine to communicate with humans has long been associated with the general success of AI. This dates back to Alan Turing's epoch-making work in the early 1950s, which proposes that a machine's intelligence can be tested…

Computation and Language · Computer Science 2020-02-03 Jiwei Li

The lack of time-efficient and reliable evaluation methods hamper the development of conversational dialogue systems (chatbots). Evaluations requiring humans to converse with chatbots are time and cost-intensive, put high cognitive demands…

Artificial Intelligence · Computer Science 2020-10-06 Jan Deriu , Don Tuggener , Pius von Däniken , Jon Ander Campos , Alvaro Rodrigo , Thiziri Belkacem , Aitor Soroa , Eneko Agirre , Mark Cieliebak

Chart question answering (CQA) is a task used for assessing chart comprehension, which is fundamentally different from understanding natural images. CQA requires analyzing the relationships between the textual and the visual components of a…

Computer Vision and Pattern Recognition · Computer Science 2022-07-12 Matan Levy , Rami Ben-Ari , Dani Lischinski

Math is constructed by people for people: just as natural language corpora reflect not just propositions but the communicative goals of language users, the math data that models are trained on reflects not just idealized mathematical…

Artificial Intelligence · Computer Science 2024-11-01 Sasha Boguraev , Ben Lipkin , Leonie Weissweiler , Kyle Mahowald

In this paper, we present the mQA model, which is able to answer questions about the content of an image. The answer can be a sentence, a phrase or a single word. Our model contains four components: a Long Short-Term Memory (LSTM) to…

Computer Vision and Pattern Recognition · Computer Science 2015-11-04 Haoyuan Gao , Junhua Mao , Jie Zhou , Zhiheng Huang , Lei Wang , Wei Xu

Intelligent personal assistant systems with either text-based or voice-based conversational interfaces are becoming increasingly popular around the world. Retrieval-based conversation models have the advantages of returning fluent and…

Information Retrieval · Computer Science 2018-05-10 Liu Yang , Minghui Qiu , Chen Qu , Jiafeng Guo , Yongfeng Zhang , W. Bruce Croft , Jun Huang , Haiqing Chen

Time plays a critical role in how information is generated, retrieved, and interpreted. In this survey, we provide a comprehensive overview of Temporal Question Answering (TQA), a research area that focuses on answering questions involving…

Computation and Language · Computer Science 2026-04-24 Bhawna Piryani , Abdelrahman Abdallah , Jamshid Mozafari , Avishek Anand , Adam Jatowt