English
Related papers

Related papers: Improving Human-Labeled Data through Dynamic Autom…

200 papers

Anomaly detection in time-series data is crucial for identifying faults, failures, threats, and outliers across a range of applications. Recently, deep learning techniques have been applied to this topic, but they often struggle in…

Machine Learning · Computer Science 2024-01-23 Lixu Wang , Shichao Xu , Xinyu Du , Qi Zhu

Imperfect labels are ubiquitous in real-world datasets. Several recent successful methods for training deep neural networks (DNNs) robust to label noise have used two primary techniques: filtering samples based on loss during a warm-up…

Computer Vision and Pattern Recognition · Computer Science 2021-04-05 Kento Nishi , Yi Ding , Alex Rich , Tobias Höllerer

Learning against label noise is a vital topic to guarantee a reliable performance for deep neural networks. Recent research usually refers to dynamic noise modeling with model output probabilities and loss values, and then separates clean…

Machine Learning · Statistics 2022-07-13 Yingsong Huang , Bing Bai , Shengwei Zhao , Kun Bai , Fei Wang

In this paper, we analyze PAC learnability from labels produced by crowdsourcing. In our setting, unlabeled examples are drawn from a distribution and labels are crowdsourced from workers who operate under classification noise, each with…

Machine Learning · Computer Science 2019-02-14 Shelby Heinecke , Lev Reyzin

Toxicity classification in textual content remains a significant problem. Data with labels from a single annotator fall short of capturing the diversity of human perspectives. Therefore, there is a growing need to incorporate crowdsourced…

Artificial Intelligence · Computer Science 2024-11-11 Zelei Cheng , Xian Wu , Jiahao Yu , Shuo Han , Xin-Qiang Cai , Xinyu Xing

This paper introduces a novel crowdsourcing worker selection algorithm, enhancing annotation quality and reducing costs. Unlike previous studies targeting simpler tasks, this study contends with the complexities of label interdependencies…

Computation and Language · Computer Science 2024-07-30 Yujie Wang , Chao Huang , Liner Yang , Zhixuan Fang , Yaping Huang , Yang Liu , Jingsi Yu , Erhong Yang

We propose a meta-learning method for learning from multiple noisy annotators. In many applications such as crowdsourcing services, labels for supervised learning are given by multiple annotators. Since the annotators have different skills…

Machine Learning · Computer Science 2025-06-13 Atsutoshi Kumagai , Tomoharu Iwata , Taishi Nishiyama , Yasutoshi Ida , Yasuhiro Fujiwara

Crowdsourcing platforms offer a practical solution to the problem of affordably annotating large datasets for training supervised classifiers. Unfortunately, poor worker performance frequently threatens to compromise annotation reliability,…

Machine Learning · Computer Science 2014-01-17 Liyue Zhao , Yu Zhang , Gita Sukthankar

Deep learning has shown remarkable success in medical image analysis, but its reliance on large volumes of high-quality labeled data limits its applicability. While noisy labeled data are easier to obtain, directly incorporating them into…

Computer Vision and Pattern Recognition · Computer Science 2025-10-21 Chengxuan Qian , Kai Han , Jianxia Ding , Chongwen Lyu , Zhenlong Yuan , Jun Chen , Zhe Liu

Large language models (LLMs) are increasingly positioned as scalable tools for annotating educational data, including classroom discourse, interaction logs, and qualitative learning artifacts. Their ability to rapidly summarize…

Artificial Intelligence · Computer Science 2026-03-17 Bakhtawar Ahtisham , Kirk Vanacore , Rene F. Kizilcec

Unsupervised Domain Adaptation (DA) is used to automatize the task of labeling data: an unlabeled dataset (target) is annotated using a labeled dataset (source) from a related domain. We cast domain adaptation as the problem of finding…

Machine Learning · Statistics 2018-03-22 Twan van Laarhoven , Elena Marchiori

We propose a new approach to address the text classification problems when learning with partial labels is beneficial. Instead of offering each training sample a set of candidate labels, we assign negative-oriented labels to the ambiguous…

Machine Learning · Statistics 2019-06-11 Jiangning Chen , Zhibo Dai , Juntao Duan , Qianli Hu , Ruilin Li , Heinrich Matzinger , Ionel Popescu , Haoyan Zhai

Deep learning models for natural language processing rely heavily on high-quality labeled datasets. However, existing labeling approaches often struggle to balance label quality with labeling cost. To address this challenge, we propose…

Human-Computer Interaction · Computer Science 2026-02-17 Guozheng Li , Ao Wang , Shaoxiang Wang , Yu Zhang , Pengcheng Cao , Yang Bai , Chi Harold Liu

Recent studies in deep learning have shown significant progress in named entity recognition (NER). Most existing works assume clean data annotation, yet a fundamental challenge in real-world scenarios is the large amount of noise from a…

Computation and Language · Computer Science 2021-04-13 Kun Liu , Yao Fu , Chuanqi Tan , Mosha Chen , Ningyu Zhang , Songfang Huang , Sheng Gao

As larger and more comprehensive datasets become standard in contemporary machine learning, it becomes increasingly more difficult to obtain reliable, trustworthy label information with which to train sophisticated models. To address this…

Machine Learning · Computer Science 2021-06-08 Glenn Dawson , Robi Polikar

Unsupervised domain adaptation (UDA) aims to transfer the knowledge from the labeled source domain to the unlabeled target domain. Existing self-training based UDA approaches assign pseudo labels for target data and treat them as ground…

Computer Vision and Pattern Recognition · Computer Science 2021-03-10 Xiaoqing Guo , Chen Yang , Baopu Li , Yixuan Yuan

We propose a novel sample selection method for image classification in the presence of noisy labels. Existing methods typically consider small-loss samples as correctly labeled. However, some correctly labeled samples are inherently…

Computer Vision and Pattern Recognition · Computer Science 2025-04-25 Weiran Pan , Wei Wei , Feida Zhu , Yong Deng

Noisy labels are inevitable in real-world scenarios. Due to the strong capacity of deep neural networks to memorize corrupted labels, these noisy labels can cause significant performance degradation. Existing research on mitigating the…

Machine Learning · Computer Science 2025-10-02 Xinlei Zhang , Fan Liu , Chuanyi Zhang , Fan Cheng , Yuhui Zheng

Deep learning has made many remarkable achievements in many fields but suffers from noisy labels in datasets. The state-of-the-art learning with noisy label method Co-teaching and Co-teaching+ confronts the noisy label by mutual-information…

Computer Vision and Pattern Recognition · Computer Science 2022-08-16 Jiarun Liu , Daguang Jiang , Yukun Yang , Ruirui Li

Computer-aided diagnosis systems must make critical decisions from medical images that are often noisy, ambiguous, or conflicting, yet today's models are trained on overly simplistic labels that ignore diagnostic uncertainty. One-hot labels…

Computer Vision and Pattern Recognition · Computer Science 2025-09-16 Ang Nan Gu , Michael Tsang , Hooman Vaseli , Purang Abolmaesumi , Teresa Tsang