English
Related papers

Related papers: Anticipating Safety Issues in E2E Conversational A…

200 papers

Large-scale language technologies are increasingly used in various forms of communication with humans across different contexts. One particular use case for these technologies is conversational agents, which output natural language text in…

Computers and Society · Computer Science 2022-12-22 Atoosa Kasirzadeh , Iason Gabriel

The main goal of modeling human conversation is to create agents which can interact with people in both open-ended and goal-oriented scenarios. End-to-end trained neural dialog systems are an important line of research for such generalized…

Computation and Language · Computer Science 2019-04-30 Chaitanya K. Joshi , Fei Mi , Boi Faltings

Open-weight advanced AI models -- systems whose parameters are freely available for download and adaptation -- are reshaping the global AI landscape. As these models rapidly close the performance gap with closed alternatives, they enable…

Computers and Society · Computer Science 2026-02-24 Bengüsu Özcan , Alex Petropoulos , Max Reddel

A long-term goal of machine learning is to build intelligent conversational agents. One recent popular approach is to train end-to-end models on a large amount of real dialog transcripts between humans (Sordoni et al., 2015; Vinyals & Le,…

Computation and Language · Computer Science 2016-04-20 Jesse Dodge , Andreea Gane , Xiang Zhang , Antoine Bordes , Sumit Chopra , Alexander Miller , Arthur Szlam , Jason Weston

Following the rapid increase in Artificial Intelligence (AI) capabilities in recent years, the AI community has voiced concerns regarding possible safety risks. To support decision-making on the safe use and development of AI systems, there…

Machine Learning · Computer Science 2025-04-01 Gil Gekker , Meirav Segal , Dan Lahav , Omer Nevo

While end-to-end neural conversation models have led to promising advances in reducing hand-crafted features and errors induced by the traditional complex system architecture, they typically require an enormous amount of data due to the…

Computation and Language · Computer Science 2018-01-10 Sungjin Lee

As language models are increasingly deployed as autonomous agents in high-stakes settings, ensuring that they reliably follow user-defined rules has become a critical safety concern. To this end, we study whether language models exhibit…

Machine Learning · Computer Science 2025-08-28 Dylan Sam , Alexander Robey , Andy Zou , Matt Fredrikson , J. Zico Kolter

Conversational AI systems exhibit a level of human-like behavior that promises to have profound impacts on many aspects of daily life -- how people access information, create content, and seek social support. Yet these models have also…

Human-Computer Interaction · Computer Science 2023-06-21 Christopher M. Homan , Greg Serapio-Garcia , Lora Aroyo , Mark Diaz , Alicia Parrish , Vinodkumar Prabhakaran , Alex S. Taylor , Ding Wang

Traditional dialog systems used in goal-oriented applications require a lot of domain-specific handcrafting, which hinders scaling up to new domains. End-to-end dialog systems, in which all components are trained from the dialogs…

Computation and Language · Computer Science 2017-04-03 Antoine Bordes , Y-Lan Boureau , Jason Weston

Learning an efficient manager of dialogue agent from data with little manual intervention is important, especially for goal-oriented dialogues. However, existing methods either take too many manual efforts (e.g. reinforcement learning…

Computation and Language · Computer Science 2019-08-16 Zhuoxuan Jiang , Xian-Ling Mao , Ziming Huang , Jie Ma , Shaochun Li

Most adversarial threats in artificial intelligence (AI) target the computational behavior of models rather than the humans who rely on them. Yet modern AI systems increasingly operate within human decision loops, where users interpret and…

Artificial Intelligence · Computer Science 2026-05-18 Shutong Fan , Lan Zhang , Xiaoyong Yuan

A core challenge in the development of increasingly capable AI systems is to make them safe and reliable by ensuring their behaviour is consistent with human values. This challenge, known as the alignment problem, does not merely apply to…

Machine Learning · Computer Science 2023-11-07 Raphaël Millière

Conversational systems or chatbots are an example of AI-Infused Applications (AIIA). Chatbots are especially important as they are often the first interaction of clients with a business and are the entry point of a business into the AI…

Large language model-based agents are rapidly evolving from simple conversational assistants into autonomous systems capable of performing complex, professional-level tasks in various domains. While these advancements promise significant…

Conversational agents (CAs) embodied in speakers or chatbots are becoming very popular in some countries, and despite their adult-centred design, they have become part of children's lives, generating a need for children-centric trustworthy…

Human-Computer Interaction · Computer Science 2022-09-07 Marina Escobar-Planas , Emilia Gómez , Carlos-D Martínez-Hinarejos

The technology of Conversational AI has made significant advancements over the last eighteen months. As a consequence, conversational agents are likely to be deployed in the near future that are designed to pursue targeted influence…

Human-Computer Interaction · Computer Science 2024-12-30 Louis Rosenberg

Dialogue safety problems severely limit the real-world deployment of neural conversational models and have attracted great research interests recently. However, dialogue safety problems remain under-defined and the corresponding dataset is…

Computation and Language · Computer Science 2022-04-05 Hao Sun , Guangxuan Xu , Jiawen Deng , Jiale Cheng , Chujie Zheng , Hao Zhou , Nanyun Peng , Xiaoyan Zhu , Minlie Huang

End-to-end (E2E) models, which directly predict output character sequences given input speech, are good candidates for on-device speech recognition. E2E models, however, present numerous challenges: In order to be truly useful, such models…

Neural models have become ubiquitous in automatic speech recognition systems. While neural networks are typically used as acoustic models in more complex systems, recent studies have explored end-to-end speech recognition systems based on…

Computation and Language · Computer Science 2017-09-15 Yonatan Belinkov , James Glass

Unlike traditional cascaded pipelines, end-to-end (E2E) spoken dialogue systems preserve full differentiability and capture non-phonemic information, making them well-suited for modeling spoken interactions. However, existing E2E approaches…

Computation and Language · Computer Science 2025-06-03 Siddhant Arora , Jinchuan Tian , Hayato Futami , Jee-weon Jung , Jiatong Shi , Yosuke Kashiwagi , Emiru Tsunoo , Shinji Watanabe