English
Related papers

Related papers: Manual Evaluation Matters: Reviewing Test Protocol…

200 papers

Benchmark datasets in computer vision often contain off-topic images, near duplicates, and label errors, leading to inaccurate estimates of model performance. In this paper, we revisit the task of data cleaning and formalize it as either a…

Relation Extraction (RE) serves as a crucial technology for transforming unstructured text into structured information, especially within the framework of Knowledge Graph development. Its importance is emphasized by its essential role in…

Computation and Language · Computer Science 2024-06-27 Dawulie Jinensibieke , Mieradilijiang Maimaiti , Wentao Xiao , Yuanhang Zheng , Xiaobo Wang

Multimodal relation extraction (MRE) is the task of identifying the semantic relationships between two entities based on the context of the sentence image pair. Existing retrieval-augmented approaches mainly focused on modeling the…

Computation and Language · Computer Science 2023-05-26 Xuming Hu , Zhijiang Guo , Zhiyang Teng , Irwin King , Philip S. Yu

We study the problem of training named entity recognition (NER) models using only distantly-labeled data, which can be automatically obtained by matching entity mentions in the raw text with entity types in a knowledge base. The biggest…

Computation and Language · Computer Science 2021-09-13 Yu Meng , Yunyi Zhang , Jiaxin Huang , Xuan Wang , Yu Zhang , Heng Ji , Jiawei Han

Analysing the generalisation capabilities of relation extraction (RE) models is crucial for assessing whether they learn robust relational patterns or rely on spurious correlations. Our cross-dataset experiments find that RE models struggle…

Computation and Language · Computer Science 2025-12-16 Varvara Arzt , Allan Hanbury , Michael Wiegand , Gábor Recski , Terra Blevins

Relation Extraction (RE) is a fundamental task of information extraction, which has attracted a large amount of research attention. Previous studies focus on extracting the relations within a sentence or document, while currently…

Computation and Language · Computer Science 2022-11-01 Fengqi Wang , Fei Li , Hao Fei , Jingye Li , Shengqiong Wu , Fangfang Su , Wenxuan Shi , Donghong Ji , Bo Cai

Machine learning methods have proven useful in transcribing historical data. However, results from even highly accurate methods require manual verification and correction. Such manual review can be time-consuming and expensive, therefore…

Machine Learning · Computer Science 2023-06-29 Bjørn-Richard Pedersen , Rigmor Katrine Johansen , Einar Holsbø , Hilde Sommerseth , Lars Ailo Bongo

The goal of open relation extraction (OpenRE) is to develop an RE model that can generalize to new relations not encountered during training. Existing studies primarily formulate OpenRE as a clustering task. They first cluster all test…

Computation and Language · Computer Science 2025-09-19 Hongyao Tu , Liang Zhang , Yujie Lin , Xin Lin , Haibo Zhang , Long Zhang , Jinsong Su

The effective extraction of ranked disease-symptom relationships is a critical component in various medical tasks, including computer-assisted medical diagnosis or the discovery of unexpected associations between diseases. While existing…

Information Retrieval · Computer Science 2020-01-16 Markus Zlabinger , Sebastian Hofstätter , Navid Rekabsaz , Allan Hanbury

Major advancements in computer vision can primarily be attributed to the use of labeled datasets. However, acquiring labels for datasets often results in errors which can harm model performance. Recent works have proposed methods to…

Computer Vision and Pattern Recognition · Computer Science 2023-12-06 Maya Srikanth , Jeremy Irvin , Brian Wesley Hill , Felipe Godoy , Ishan Sabane , Andrew Y. Ng

Relation extraction (RE) is the core NLP task of inferring semantic relationships between entities from text. Standard supervised RE techniques entail training modules to tag tokens comprising entity spans and then predict the relationship…

Computation and Language · Computer Science 2024-07-17 Somin Wadhwa , Silvio Amir , Byron C. Wallace

Distant supervision significantly reduces human efforts in building training data for many classification tasks. While promising, this technique often introduces noise to the generated training data, which can severely affect the model…

Computation and Language · Computer Science 2018-05-16 Bingfeng Luo , Yansong Feng , Zheng Wang , Zhanxing Zhu , Songfang Huang , Rui Yan , Dongyan Zhao

Quality of data plays an important role in most deep learning tasks. In the speech community, transcription of speech recording is indispensable. Since the transcription is usually generated artificially, automatically finding errors in…

Computation and Language · Computer Science 2019-07-23 Xiaofei Wang , Jinyi Yang , Ruizhi Li , Samik Sadhu , Hynek Hermansky

Distant supervision for relation extraction provides uniform bag labels for each sentence inside the bag, while accurate sentence labels are important for downstream applications that need the exact relation type. Directly using bag labels…

Computation and Language · Computer Science 2021-06-23 Ruotian Ma , Tao Gui , Linyang Li , Qi Zhang , Yaqian Zhou , Xuanjing Huang

In this work, we present a Web-based annotation tool `Relation Triplets Extractor' \footnote{https://abera87.github.io/annotate/} (RTE) for annotating relation triplets from the text. Relation extraction is an important task for extracting…

Computation and Language · Computer Science 2021-08-19 Ankan Mullick , Animesh Bera , Tapas Nayak

Recent semi-supervised learning methods use pseudo supervision as core idea, especially self-training methods that generate pseudo labels. However, pseudo labels are unreliable. Self-training methods usually rely on single model prediction…

Computer Vision and Pattern Recognition · Computer Science 2022-05-12 Zhengyang Feng , Qianyu Zhou , Qiqi Gu , Xin Tan , Guangliang Cheng , Xuequan Lu , Jianping Shi , Lizhuang Ma

Document-level relation extraction faces two overlooked challenges: long-tail problem and multi-label problem. Previous work focuses mainly on obtaining better contextual representations for entity pairs, hardly address the above…

Computation and Language · Computer Science 2022-12-21 Ridong Han , Tao Peng , Benyou Wang , Lu Liu , Xiang Wan

Automatic extraction of cause-effect relationships from natural language texts is a challenging open problem in Artificial Intelligence. Most of the early attempts at its solution used manually constructed linguistic and syntactic rules on…

Artificial Intelligence · Computer Science 2016-05-26 Nabiha Asghar

Document-level relation extraction (DocRE) aims to extract semantic relations among entity pairs in a document. Typical DocRE methods blindly take the full document as input, while a subset of the sentences in the document, noted as the…

Computation and Language · Computer Science 2022-03-08 Yiqing Xie , Jiaming Shen , Sha Li , Yuning Mao , Jiawei Han

Meta-analyses statistically aggregate the findings of different randomized controlled trials (RCTs) to assess treatment effectiveness. Because this yields robust estimates of treatment effectiveness, results from meta-analyses are…

Computation and Language · Computer Science 2024-07-26 Hye Sun Yun , David Pogrebitskiy , Iain J. Marshall , Byron C. Wallace