English

Characterizing Datasets for LLM-based Requirements Engineering: A Systematic Mapping Study

Software Engineering 2026-04-23 v2

Abstract

Large Language Models (LLMs) depend on high-quality, domain-specific natural language datasets. This dependency is particularly pronounced in Requirements Engineering (RE), where core activities rely on textual artifacts such as requirements, specifications, and stakeholder feedback. Despite the increasing use of LLMs in RE, data scarcity remains a widely reported limitation. While several datasets support LLM-based RE research, they are scattered across studies and lack systematic characterization, hindering reuse, comparability and assessment. This paper addresses this gap by examining which public datasets are used in LLM-based RE, how they can be consistently characterized, and which RE tasks and dataset properties remain under-represented. We report on a systematic mapping study of 45 primary studies referencing 62 publicly available datasets. Each dataset is characterized using a structured scheme covering multiple dimensions, including relevant descriptors such as artifact type, granularity, RE activity, supported task, application domain, and language, among others. The results reveal notable imbalances, including an incomplete adoption of open-science practices, limited dataset support for elicitation activities, and a lack of language and socio-technical diversity. The resulting catalogue and characterisation scheme support informed dataset selection, comparison, and reuse, contributing to stronger empirical foundations for LLM-based RE research and evaluation.

Keywords

Cite

@article{arxiv.2510.18787,
  title  = {Characterizing Datasets for LLM-based Requirements Engineering: A Systematic Mapping Study},
  author = {Quim Motger and Carlota Catot and Xavier Franch},
  journal= {arXiv preprint arXiv:2510.18787},
  year   = {2026}
}

Comments

Accepted at the 30th International Conference on Evaluation and Assessment in Software Engineering

R2 v1 2026-07-01T06:58:11.624Z