Overview of the SV-Ident 2022 Shared Task on Survey Variable Identification in Social Science Publications

Tornike Tsereteli; Yavuz Selim Kartal; Simone Paolo Ponzetto; Andrea Zielinski; Kai Eckert; Philipp Mayr

Overview of the SV-Ident 2022 Shared Task on Survey Variable Identification in Social Science Publications

Computation and Language 2022-09-20 v1 Information Retrieval

Authors: Tornike Tsereteli , Yavuz Selim Kartal , Simone Paolo Ponzetto , Andrea Zielinski , Kai Eckert , Philipp Mayr

Abstract

In this paper, we provide an overview of the SV-Ident shared task as part of the 3rd Workshop on Scholarly Document Processing (SDP) at COLING 2022. In the shared task, participants were provided with a sentence and a vocabulary of variables, and asked to identify which variables, if any, are mentioned in individual sentences from scholarly documents in full text. Two teams made a total of 9 submissions to the shared task leaderboard. While none of the teams improve on the baseline systems, we still draw insights from their submissions. Furthermore, we provide a detailed evaluation. Data and baselines for our shared task are freely available at https://github.com/vadis-project/sv-ident

Keywords

survey variable identification shared task scholarly documents social science sv-ident baseline systems

Cite

@article{arxiv.2209.09062,
  title  = {Overview of the SV-Ident 2022 Shared Task on Survey Variable Identification in Social Science Publications},
  author = {Tornike Tsereteli and Yavuz Selim Kartal and Simone Paolo Ponzetto and Andrea Zielinski and Kai Eckert and Philipp Mayr},
  journal= {arXiv preprint arXiv:2209.09062},
  year   = {2022}
}

Overview of the SV-Ident 2022 Shared Task on Survey Variable Identification in Social Science Publications

Abstract

Keywords

Cite

Related papers