Understanding how software developers think, make decisions, and behave remains a key challenge in software engineering (SE). Verbalization techniques (methods that capture spoken or written thought processes) offer a lightweight and accessible way to study these cognitive aspects. This paper presents a scoping review of research at the intersection of SE and psychology (PSY), focusing on the use of verbal data. To make large-scale interdisciplinary reviews feasible, we employed a large language model (LLM)-assisted screening pipeline using GPT to assess the relevance of over 9,000 papers based solely on titles. We addressed two questions: what themes emerge from verbalization-related work in SE, and how effective are LLMs in supporting interdisciplinary review processes? We validated GPT's outputs against human reviewers and found high consistency, with a 13\% disagreement rate. Prominent themes mainly were tied to the craft of SE, while more human-centered topics were underrepresented. The data also suggests that SE frequently draws on PSY methods, whereas the reverse is rare.
@article{arxiv.2510.12294,
title = {Show Your Title! A Scoping Review on Verbalization in Software Engineering with LLM-Assisted Screening},
author = {Gergő Balogh and Dávid Kószó and Homayoun Safarpour Motealegh Mahalegi and László Tóth and Bence Szakács and Áron Búcsú},
journal= {arXiv preprint arXiv:2510.12294},
year = {2025}
}
Comments
preprint of a paper under publication in Quality of Information and Communications Technology 2025