Improving astroBERT using Semantic Textual Similarity
Computation and Language
2022-12-02 v1 Instrumentation and Methods for Astrophysics
Abstract
The NASA Astrophysics Data System (ADS) is an essential tool for researchers that allows them to explore the astronomy and astrophysics scientific literature, but it has yet to exploit recent advances in natural language processing. At ADASS 2021, we introduced astroBERT, a machine learning language model tailored to the text used in astronomy papers in ADS. In this work we: - announce the first public release of the astroBERT language model; - show how astroBERT improves over existing public language models on astrophysics specific tasks; - and detail how ADS plans to harness the unique structure of scientific papers, the citation graph and citation context, to further improve astroBERT.
Cite
@article{arxiv.2212.00744,
title = {Improving astroBERT using Semantic Textual Similarity},
author = {Felix Grezes and Thomas Allen and Sergi Blanco-Cuaresma and Alberto Accomazzi and Michael J. Kurtz and Golnaz Shapurian and Edwin Henneken and Carolyn S. Grant and Donna M. Thompson and Timothy W. Hostetler and Matthew R. Templeton and Kelly E. Lockhart and Shinyi Chen and Jennifer Koch and Taylor Jacovich and Pavlos Protopapas},
journal= {arXiv preprint arXiv:2212.00744},
year = {2022}
}