English
Related papers

Related papers: Noise-Robust De-Duplication at Scale

200 papers

Real-world image noise removal is a long-standing yet very challenging task in computer vision. The success of deep neural network in denoising stimulates the research of noise generation, aiming at synthesizing more clean-noisy image pairs…

Computer Vision and Pattern Recognition · Computer Science 2020-07-14 Zongsheng Yue , Qian Zhao , Lei Zhang , Deyu Meng

Neural network-based speaker recognition has achieved significant improvement in recent years. A robust speaker representation learns meaningful knowledge from both hard and easy samples in the training set to achieve good performance.…

Audio and Speech Processing · Electrical Eng. & Systems 2022-10-31 Ruijie Tao , Kong Aik Lee , Zhan Shi , Haizhou Li

Identifying the veracity of a news article is an interesting problem while automating this process can be a challenging task. Detection of a news article as fake is still an open question as it is contingent on many factors which the…

Computation and Language · Computer Science 2017-12-12 Gaurav Bhatt , Aman Sharma , Shivam Sharma , Ankush Nagpal , Balasubramanian Raman , Ankush Mittal

Recently, deep learning models have been widely applied in program understanding tasks, and these models achieve state-of-the-art results on many benchmark datasets. A major challenge of deep learning for program understanding is that the…

Software Engineering · Computer Science 2024-01-02 Wenhan Wang , Yanzhou Li , Anran Li , Jian Zhang , Wei Ma , Yang Liu

In this paper, we propose SCANING, an unsupervised framework for paraphrasing via controlled noise injection. We focus on the novel task of paraphrasing algebraic word problems having practical applications in online pedagogy as a means to…

Computation and Language · Computer Science 2023-02-07 Rishabh Gupta , Venktesh V. , Mukesh Mohania , Vikram Goyal

Data duplication during pretraining can degrade generalization and lead to memorization, motivating aggressive deduplication pipelines. However, at web scale, it is unclear what constitutes a ``duplicate'': beyond surface-form matches,…

Machine Learning · Computer Science 2026-03-10 Joshua Kazdan , Noam Levi , Rylan Schaeffer , Jessica Chudnovsky , Abhay Puri , Bo He , Mehmet Donmez , Sanmi Koyejo , David Donoho

Data deduplication saves storage space by identifying and removing repeats in the data stream. Compared with traditional compression methods, data deduplication schemes are more time efficient and are thus widely used in large scale storage…

Information Theory · Computer Science 2022-05-30 Hao Lou , Farzad Farnoud

Neural text generation (data- or text-to-text) demonstrates remarkable performance when training data is abundant which for many applications is not the case. To collect a large corpus of parallel data, heuristic rules are often used but…

Computation and Language · Computer Science 2020-10-13 Katja Filippova

Existing self-supervised pre-trained speech models have offered an effective way to leverage massive unannotated corpora to build good automatic speech recognition (ASR). However, many current models are trained on a clean corpus from a…

Sound · Computer Science 2023-03-01 Dianwen Ng , Ruixi Zhang , Jia Qi Yip , Zhao Yang , Jinjie Ni , Chong Zhang , Yukun Ma , Chongjia Ni , Eng Siong Chng , Bin Ma

Deep neural networks have proven to be highly effective when large amounts of data with clean labels are available. However, their performance degrades when training data contains noisy labels, leading to poor generalization on the test…

Computer Vision and Pattern Recognition · Computer Science 2023-08-15 Fahimeh Fooladgar , Minh Nguyen Nhat To , Parvin Mousavi , Purang Abolmaesumi

Named entity recognition (NER) models often struggle with noisy inputs, such as those with spelling mistakes or errors generated by Optical Character Recognition processes, and learning a robust NER model is challenging. Existing robust NER…

Computation and Language · Computer Science 2024-07-29 Chaoyi Ai , Yong Jiang , Shen Huang , Pengjun Xie , Kewei Tu

This paper introduces a highly efficient greedy decoding algorithm for Transducer-based speech recognition models. We redesign the standard nested-loop design for RNN-T decoding, swapping loops over frames and labels: the outer loop…

Audio and Speech Processing · Electrical Eng. & Systems 2025-01-20 Vladimir Bataev , Hainan Xu , Daniel Galvez , Vitaly Lavrukhin , Boris Ginsburg

Graph neural networks (GNNs) have been widely used in various graph machine learning scenarios. Existing literature primarily assumes well-annotated training graphs, while the reliability of labels is not guaranteed in real-world scenarios.…

Machine Learning · Computer Science 2026-01-27 Yusheng Zhao , Jiaye Xie , Qixin Zhang , Weizhi Zhang , Xiao Luo , Zhiping Xiao , Philip S. Yu , Ming Zhang

Recent advances in Deep Learning have led to a significant performance increase on several NLP tasks, however, the models become more and more computationally demanding. Therefore, this paper tackles the domain of computationally efficient…

Computation and Language · Computer Science 2022-05-18 Pedro Alonso , Kumar Shridhar , Denis Kleyko , Evgeny Osipov , Marcus Liwicki

The exercise of detecting similar bug reports in bug tracking systems is known as duplicate bug report detection. Having prior knowledge of a bug report's existence reduces efforts put into debugging problems and identifying the root cause.…

Software Engineering · Computer Science 2023-11-15 Avinash Patil , Aryan Jadon

Large web-crawled corpora represent an excellent resource for improving the performance of Neural Machine Translation (NMT) systems across several language pairs. However, since these corpora are typically extremely noisy, their use is…

Computation and Language · Computer Science 2021-03-15 Gaurav Kumar , Philipp Koehn , Sanjeev Khudanpur

Fake News on social media platforms has attracted a lot of attention in recent times, primarily for events related to politics (2016 US Presidential elections), healthcare (infodemic during COVID-19), to name a few. Various methods have…

Computation and Language · Computer Science 2022-11-28 Mohit Mayank , Shakshi Sharma , Rajesh Sharma

Collecting large-scale data with clean labels for supervised training of neural networks is practically challenging. Although noisy labels are usually cheap to acquire, existing methods suffer a lot from label noise. This paper targets at…

Machine Learning · Computer Science 2020-06-16 Zizhao Zhang , Han Zhang , Sercan O. Arik , Honglak Lee , Tomas Pfister

Noisy labels are ubiquitous in real-world datasets, especially in the large-scale ones derived from crowdsourcing and web searching. It is challenging to train deep neural networks with noisy datasets since the networks are prone to…

Computer Vision and Pattern Recognition · Computer Science 2024-06-26 Yangdi Lu , Wenbo He

The proliferation of online news enables potential widespread publication of perceived low-quality news headlines/links. As a result, we investigated whether it was possible to automatically distinguish perceived lower-quality news…

Computation and Language · Computer Science 2025-06-12 Austin McCutcheon , Thiago E. A. de Oliveira , Aleksandr Zheleznov , Chris Brogly