English
Related papers

Related papers: Investigating Annotator Bias in Large Language Mod…

200 papers

Large Language Models (LLMs) demonstrate impressive capabilities across various fields, yet their increasing use raises critical security concerns. This article reviews recent literature addressing key issues in LLM security, with a focus…

Cryptography and Security · Computer Science 2025-11-26 Benji Peng , Keyu Chen , Ming Li , Pohsun Feng , Ziqian Bi , Junyu Liu , Xinyuan Song , Qian Niu

Political misinformation poses significant challenges to democratic processes, shaping public opinion and trust in media. Manual fact-checking methods face issues of scalability and annotator bias, while machine learning models require…

Computation and Language · Computer Science 2024-11-11 Veronica Chatrath , Marcelo Lotif , Shaina Raza

Automatic detection of hate and abusive language is essential to combat its online spread. Moreover, recognising and explaining hate speech serves to educate people about its negative effects. However, most current detection models operate…

Computation and Language · Computer Science 2025-05-06 Paloma Piot , Javier Parapar

Large Language Models (LLMs) have excelled at language understanding and generating human-level text. However, even with supervised training and human alignment, these LLMs are susceptible to adversarial attacks where malicious users can…

Computation and Language · Computer Science 2024-08-08 Shachi H Kumar , Saurav Sahay , Sahisnu Mazumder , Eda Okur , Ramesh Manuvinakurike , Nicole Beckage , Hsuan Su , Hung-yi Lee , Lama Nachman

Despite the extensive communication benefits offered by social media platforms, numerous challenges must be addressed to ensure user safety. One of the most significant risks faced by users on these platforms is targeted hate speech. Social…

Computation and Language · Computer Science 2024-07-18 Sadar Jaf , Basel Barakat

Warning: this paper contains content that may be offensive or upsetting Hate speech moderation on global platforms poses unique challenges due to the multimodal and multilingual nature of content, along with the varying cultural…

Computation and Language · Computer Science 2025-02-18 Minh Duc Bui , Katharina von der Wense , Anne Lauscher

Annotated data plays a critical role in Natural Language Processing (NLP) in training models and evaluating their performance. Given recent developments in Large Language Models (LLMs), models such as ChatGPT demonstrate zero-shot…

Computation and Language · Computer Science 2024-03-18 Minzhi Li , Taiwei Shi , Caleb Ziems , Min-Yen Kan , Nancy F. Chen , Zhengyuan Liu , Diyi Yang

Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere. Despite this success, these models can…

Computation and Language · Computer Science 2024-07-16 Isabel O. Gallegos , Ryan A. Rossi , Joe Barrow , Md Mehrab Tanjim , Sungchul Kim , Franck Dernoncourt , Tong Yu , Ruiyi Zhang , Nesreen K. Ahmed

Detecting hateful content is a challenging and important problem. Automated tools, like machine-learning models, can help, but they require continuous training to adapt to the ever-changing landscape of social media. In this work, we…

Computation and Language · Computer Science 2025-11-06 Jay Patel , Hrudayangam Mehta , Jeremy Blackburn

Migration has been a core topic in German political debate, from the postwar displacement of millions of expellees to labor migration and recent refugee movements. Studying political speech across such wide-ranging phenomena in depth has…

Computation and Language · Computer Science 2026-04-06 Aida Kostikova , Ole Pütz , Steffen Eger , Olga Sabelfeld , Benjamin Paassen

Large language models (LLMs) have become ubiquitous, thus it is important to understand their risks and limitations. Smaller LLMs can be deployed where compute resources are constrained, such as edge devices, but with different propensity…

Computation and Language · Computer Science 2025-04-22 Berk Atil , Vipul Gupta , Sarkar Snigdha Sarathi Das , Rebecca J. Passonneau

With the proliferation of social media, there has been a sharp increase in offensive content, particularly targeting vulnerable groups, exacerbating social problems such as hatred, racism, and sexism. Detecting offensive language use is…

Computation and Language · Computer Science 2023-12-05 Toygar Tanyel , Besher Alkurdi , Serkan Ayvaz

In current hate speech datasets, there exists a high correlation between annotators' perceptions of toxicity and signals of African American English (AAE). This bias in annotated training data and the tendency of machine learning models to…

Computation and Language · Computer Science 2020-05-26 Mengzhou Xia , Anjalie Field , Yulia Tsvetkov

Sentiment analysis focuses on identifying the emotional polarity expressed in textual data, typically categorized as positive, negative, or neutral. Hate speech detection, on the other hand, aims to recognize content that incites violence,…

Computation and Language · Computer Science 2026-01-07 Meysam Shirdel Bilehsavar , Negin Mahmoudi , Mohammad Jalili Torkamani , Kiana Kiashemshaki

Language serves as a powerful tool for the manifestation of societal belief systems. In doing so, it also perpetuates the prevalent biases in our society. Gender bias is one of the most pervasive biases in our society and is seen in online…

Computation and Language · Computer Science 2023-10-27 Rishav Hada , Agrima Seth , Harshita Diddee , Kalika Bali

Today, the internet is an integral part of our daily lives, enabling people to be more connected than ever before. However, this greater connectivity and access to information increase exposure to harmful content such as cyber-bullying and…

Social and Information Networks · Computer Science 2023-10-31 Lanqin Yuan , Tianyu Wang , Gabriela Ferraro , Hanna Suominen , Marian-Andrei Rizoiu

As AI systems become more integrated into daily life, the need for safer and more reliable moderation has never been greater. Large Language Models (LLMs) have demonstrated remarkable capabilities, surpassing earlier models in complexity…

Artificial Intelligence · Computer Science 2026-01-13 Naseem Machlovi , Maryam Saleki , Innocent Ababio , Ruhul Amin

Large Language Models (LLMs) are prone to generating content that exhibits gender biases, raising significant ethical concerns. Alignment, the process of fine-tuning LLMs to better align with desired behaviors, is recognized as an effective…

Computation and Language · Computer Science 2024-12-17 Tao Zhang , Ziqian Zeng , Yuxiang Xiao , Huiping Zhuang , Cen Chen , James Foulds , Shimei Pan

Large Language Models (LLM) have made significant advances in the recent past becoming more mainstream in Artificial Intelligence (AI) enabled human-facing applications. However, LLMs often generate stereotypical output inherited from…

Computation and Language · Computer Science 2023-11-27 Wu Zekun , Sahan Bulathwela , Adriano Soares Koshiyama

Human label variation has been established as a central phenomenon in NLP: the perspectives different annotators have on the same item need to be embraced. Data collection practices thus shifted towards increasing the annotator numbers and…

Computation and Language · Computer Science 2026-05-08 Maximilian Maurer , Maximilian Linde , Gabriella Lapesa