Span-based Joint Entity and Relation Extraction with Transformer Pre-training
Abstract
We introduce SpERT, an attention model for span-based joint entity and relation extraction. Our key contribution is a light-weight reasoning on BERT embeddings, which features entity recognition and filtering, as well as relation classification with a localized, marker-free context representation. The model is trained using strong within-sentence negative samples, which are efficiently extracted in a single BERT pass. These aspects facilitate a search over all spans in the sentence. In ablation studies, we demonstrate the benefits of pre-training, strong negative sampling and localized context. Our model outperforms prior work by up to 2.6% F1 score on several datasets for joint entity and relation extraction.
Cite
@article{arxiv.1909.07755,
title = {Span-based Joint Entity and Relation Extraction with Transformer Pre-training},
author = {Markus Eberts and Adrian Ulges},
journal= {arXiv preprint arXiv:1909.07755},
year = {2021}
}
Comments
Published at ECAI 2020; marginally revised version; because of new insights into evaluation metrics used in related work, we updated Table 1 and report both micro/macro averaged entity values for the ADE dataset