English
Related papers

Related papers: Real-Time Toxicity Filtering for Open-Source Code …

200 papers

Large language models (LLMs) are increasingly popular but are also prone to generating bias, toxic or harmful language, which can have detrimental effects on individuals and communities. Although most efforts is put to assess and mitigate…

Computation and Language · Computer Science 2024-06-26 Caroline Brun , Vassilina Nikoulina

Timely assessment of compound toxicity is one of the biggest challenges facing the pharmaceutical industry today. A significant proportion of compounds identified as potential leads are ultimately discarded due to the toxicity they induce.…

Machine Learning · Statistics 2018-06-13 Mikhail Zaslavskiy , Simon Jégou , Eric W. Tramel , Gilles Wainrib

Toxic language is difficult to define, as it is not monolithic and has many variations in perceptions of toxicity. This challenge of detecting toxic language is increased by the highly contextual and subjectivity of its interpretation,…

Computation and Language · Computer Science 2023-05-19 Huriyyah Althunayan , Rahaf Bahlas , Manar Alharbi , Lena Alsuwailem , Abeer Aldayel , Rehab ALahmadi

Code generation large language models (LLMs) are increasingly integrated into modern software development workflows. Recent work has shown that these models are vulnerable to backdoor and poisoning attacks that induce the generation of…

Cryptography and Security · Computer Science 2026-03-19 Shenao Yan , Shimaa Ahmed , Shan Jin , Sunpreet S. Arora , Yiwei Cai , Yizhen Wang , Yuan Hong

Existing Chinese toxic content detection methods mainly target sentence-level classification but often fail to provide readable and contiguous toxic evidence spans. We propose \textbf{ToxiTrace}, an explainability-oriented method for…

Computation and Language · Computer Science 2026-04-15 Boyang Li , Hongzhe Shou , Yuanyuan Liang , Jingbin Zhang , Fang Zhou

Recent breakthroughs in Large Language Models (LLMs) have revealed remarkable generative capabilities and emerging self-regulatory mechanisms, including self-correction and self-rewarding. However, current detoxification techniques rarely…

Computation and Language · Computer Science 2026-01-21 Kaituo Zhang , Zhimeng Jiang , Na Zou

In this work, we demonstrate how existing classifiers for identifying toxic comments online fail to generalize to the diverse concerns of Internet users. We survey 17,280 participants to understand how user expectations for what constitutes…

Social and Information Networks · Computer Science 2021-06-09 Deepak Kumar , Patrick Gage Kelley , Sunny Consolvo , Joshua Mason , Elie Bursztein , Zakir Durumeric , Kurt Thomas , Michael Bailey

Identifying security issues early is encouraged to reduce the latent negative impacts on software systems. Code review is a widely-used method that allows developers to manually inspect modified code, catching security issues during a…

Software Engineering · Computer Science 2024-05-10 Wachiraphan Charoenwet , Patanamon Thongtanunam , Van-Thuan Pham , Christoph Treude

To identify and classify toxic online commentary, the modern tools of data science transform raw text into key features from which either thresholding or learning algorithms can make predictions for monitoring offensive conversations. We…

Machine Learning · Computer Science 2018-10-05 David Noever

With adversarial or otherwise normal prompts, existing large language models (LLM) can be pushed to generate toxic discourses. One way to reduce the risk of LLMs generating undesired discourses is to alter the training of the LLM. This can…

Computation and Language · Computer Science 2023-02-28 Meng Cao , Mehdi Fatemi , Jackie Chi Kit Cheung , Samira Shabanian

Large Language Models (LLMs) have become integral to Software Engineering (SE), increasingly used in development workflows. However, their widespread adoption raises concerns about the presence and propagation of toxic language - harmful or…

Machine Learning · Computer Science 2026-01-21 Hao Zhuo , Yicheng Yang , Kewen Peng

In this work, we introduce our solution for the Multilingual Text Detoxification Task in the PAN-2025 competition for the ylmmcl team: a robust multilingual text detoxification pipeline that integrates lexicon-guided tagging, a fine-tuned…

Computation and Language · Computer Science 2025-07-28 Nicole Lai-Lopez , Lusha Wang , Su Yuan , Liza Zhang

Real-time toxicity detection in online environments poses a significant challenge, due to the increasing prevalence of social media and gaming platforms. We introduce ToxBuster, a simple and scalable model that reliably detects toxic…

Computation and Language · Computer Science 2024-08-22 Zachary Yang , Nicolas Grenan-Godbout , Reihaneh Rabbany

Large language models (LLMs) have become integral to various real-world applications, leveraging massive, web-sourced datasets like Common Crawl, C4, and FineWeb for pretraining. While these datasets provide linguistic data essential for…

Computation and Language · Computer Science 2025-08-14 Sai Krishna Mendu , Harish Yenala , Aditi Gulati , Shanu Kumar , Parag Agrawal

Toxic language detection systems often falsely flag text that contains minority group mentions as toxic, as those groups are often the targets of online hate. Such over-reliance on spurious correlations also causes systems to struggle with…

Computation and Language · Computer Science 2022-07-15 Thomas Hartvigsen , Saadia Gabriel , Hamid Palangi , Maarten Sap , Dipankar Ray , Ece Kamar

Detecting toxicity in online spaces is challenging and an ever more pressing problem given the increase in social media and gaming consumption. We introduce ToxBuster, a simple and scalable model trained on a relatively large dataset of…

Computation and Language · Computer Science 2023-05-24 Zachary Yang , Yasmine Maricar , MohammadReza Davari , Nicolas Grenon-Godbout , Reihaneh Rabbany

Peer review is crucial for advancing and improving science through constructive criticism. However, toxic feedback can discourage authors and hinder scientific progress. This work explores an important but underexplored area: detecting…

Computation and Language · Computer Science 2025-02-05 Man Luo , Bradley Peterson , Rafael Gan , Hari Ramalingame , Navya Gangrade , Ariadne Dimarogona , Imon Banerjee , Phillip Howard

Motivation: Code understandability is crucial in software development, as developers spend 58% to 70% of their time reading source code. Improving it can improve productivity and reduce maintenance costs. Problem: Experimental studies often…

Software Engineering · Computer Science 2024-11-13 Delano Oliveira , Reydne Santos , Benedito de Oliveira , Martin Monperrus , Fernando Castor , Fernanda Madeiral

Prior works on detoxification are scattered in the sense that they do not cover all aspects of detoxification needed in a real-world scenario. Notably, prior works restrict the task of developing detoxification models to only a seen subset…

Machine Learning · Computer Science 2024-10-07 Md Tawkat Islam Khondaker , Muhammad Abdul-Mageed , Laks V. S. Lakshmanan

Toxic language is one of the major barrier to safe online participation, yet robust mitigation tools are scarce for African languages. This study addresses this critical gap by investigating automatic text detoxification (toxic to neutral…

Computation and Language · Computer Science 2026-01-12 Abayomi O. Agbeyangi