English
Related papers

Related papers: Using In-Context Learning to Improve Dialogue Safe…

200 papers

Dialogue systems in the form of chatbots and personal assistants are being increasingly integrated into people's lives. Modern dialogue systems may consider adopting anthropomorphic personas, mimicking societal demographic groups to appear…

Computation and Language · Computer Science 2021-12-16 Emily Sheng , Josh Arnold , Zhou Yu , Kai-Wei Chang , Nanyun Peng

Training dialog policies for speech-based virtual assistants requires a plethora of conversational data. The data collection phase is often expensive and time consuming due to human involvement. To address this issue, a common solution is…

Computation and Language · Computer Science 2019-11-11 Maryam Fazel-Zarandi , Longshaokan Wang , Aditya Tiwari , Spyros Matsoukas

In conversational search, agents can interact with users by asking clarifying questions to increase their chance to find better results. Many recent works and shared tasks in both NLP and IR communities have focused on identifying the need…

Information Retrieval · Computer Science 2022-01-04 Zhenduo Wang , Qingyao Ai

Despite the multi-turn open-domain dialogue systems have attracted more and more attention and made great progress, the existing dialogue systems are still very boring. Nearly all the existing dialogue models only provide a response when…

Computation and Language · Computer Science 2019-12-23 Tian Lan , Xianling Mao , Heyan Huang , Wei Wei

Human dialogues are scenario-based and appropriate responses generally relate to the latent context knowledge entailed by the specific scenario. To enable responses that are more meaningful and context-specific, we propose to improve…

Computation and Language · Computer Science 2020-10-07 Shaoxiong Feng , Xuancheng Ren , Hongshen Chen , Bin Sun , Kan Li , Xu Sun

An important aspect of developing conversational agents is to give a bot the ability to improve through communicating with humans and to learn from the mistakes that it makes. Most research has focused on learning from fixed training sets…

Artificial Intelligence · Computer Science 2017-01-17 Jiwei Li , Alexander H. Miller , Sumit Chopra , Marc'Aurelio Ranzato , Jason Weston

AI-driven chatbots have become an emerging solution to address psychological distress. Due to the lack of psychotherapeutic data, researchers use dialogues scraped from online peer support forums to train them. But since the responses in…

Computation and Language · Computer Science 2023-05-18 Anuradha Welivita , Pearl Pu

Over the last several years, end-to-end neural conversational agents have vastly improved in their ability to carry a chit-chat conversation with humans. However, these models are often trained on large datasets from the internet, and as a…

Computation and Language · Computer Science 2021-07-26 Emily Dinan , Gavin Abercrombie , A. Stevie Bergman , Shannon Spruit , Dirk Hovy , Y-Lan Boureau , Verena Rieser

The detection of offensive language in the context of a dialogue has become an increasingly important application of natural language processing. The detection of trolls in public forums (Gal\'an-Garc\'ia et al., 2016), and the deployment…

Computation and Language · Computer Science 2019-08-20 Emily Dinan , Samuel Humeau , Bharath Chintagunta , Jason Weston

Text generation has a fundamental limitation almost by definition: there is no taking back tokens that have been generated, even when they are clearly problematic. In the context of language model safety, when a partial unsafe generation is…

Machine Learning · Computer Science 2024-09-24 Yiming Zhang , Jianfeng Chi , Hailey Nguyen , Kartikeya Upasani , Daniel M. Bikel , Jason Weston , Eric Michael Smith

Multi-modal Large Language Models (MLLMs) have achieved remarkable performance across a wide range of visual reasoning tasks, yet their vulnerability to safety risks remains a pressing concern. While prior research primarily focuses on…

Computer Vision and Pattern Recognition · Computer Science 2026-03-18 Ce Zhang , Jinxi He , Junyi He , Katia Sycara , Yaqi Xie

Building dialogue systems requires a large corpus of annotated dialogues. Such datasets are usually created via crowdsourcing, which is expensive and time-consuming. In this paper, we propose \textsc{Dialogic}, a novel dialogue simulation…

Computation and Language · Computer Science 2023-06-07 Zekun Li , Wenhu Chen , Shiyang Li , Hong Wang , Jing Qian , Xifeng Yan

As large language models (LLMs) are increasingly deployed in high-stakes settings, the risk of generating harmful or toxic content remains a central challenge. Post-hoc alignment methods are brittle: once unsafe patterns are learned during…

Dialogue models are able to generate coherent and fluent responses, but they can still be challenging to control and may produce non-engaging, unsafe results. This unpredictability diminishes user trust and can hinder the use of the models…

Computation and Language · Computer Science 2023-05-23 Prakhar Gupta , Yang Liu , Di Jin , Behnam Hedayatnia , Spandana Gella , Sijia Liu , Patrick Lange , Julia Hirschberg , Dilek Hakkani-Tur

Current open-domain conversational models can easily be made to talk in inadequate ways. Online learning from conversational feedback given by the conversation partner is a promising avenue for a model to improve and adapt, so as to…

Computation and Language · Computer Science 2022-05-06 Megan Ung , Jing Xu , Y-Lan Boureau

Because state-of-the-art language models are expensive to train, most practitioners must make use of one of the few publicly available language models or language model APIs. This consolidation of trust increases the potency of backdoor…

Cryptography and Security · Computer Science 2023-07-28 Nikhil Kandpal , Matthew Jagielski , Florian Tramèr , Nicholas Carlini

Dialogue systems, commonly known as chatbots, have gained escalating popularity in recent times due to their wide-spread applications in carrying out chit-chat conversations with users and task-oriented dialogues to accomplish various user…

Computation and Language · Computer Science 2024-06-18 Sahisnu Mazumder , Bing Liu

We present a novel response generation system that can be trained end to end on large quantities of unstructured Twitter conversations. A neural network architecture is used to address sparsity issues that arise when integrating contextual…

Computation and Language · Computer Science 2015-06-23 Alessandro Sordoni , Michel Galley , Michael Auli , Chris Brockett , Yangfeng Ji , Margaret Mitchell , Jian-Yun Nie , Jianfeng Gao , Bill Dolan

Using a sequence-to-sequence framework, many neural conversation models for chit-chat succeed in naturalness of the response. Nevertheless, the neural conversation models tend to give generic responses which are not specific to given…

Computation and Language · Computer Science 2018-05-24 Jonggu Kim , Doyeon Kong , Jong-Hyeok Lee

While Large Language Models (LLMs) have shown significant advancements in performance, various jailbreak attacks have posed growing safety and ethical risks. Malicious users often exploit adversarial context to deceive LLMs, prompting them…

Cryptography and Security · Computer Science 2025-08-15 Jinhwa Kim , Ian G. Harris