相关论文: Researcher Bias in Software Engineering Experiment…
Risk-based approaches to governance bear an ambiguous stance regarding the Research and Development stages of AI, for they the possibility of explicit risks before they are posed by a given finalised product. In this context, Institutional…
Randomized benchmarking (RB) protocols are standard tools for characterizing quantum devices. Prior analyses of RB protocols have not provided a complete method for analyzing realistic data, resulting in a variety of ad-hoc methods. The…
Randomized benchmarking (RB) is a widely used strategy to assess the quality of available quantum gates in a computational context. RB involves applying known random sequences of gates to an initial state and using the statistics of a final…
Context: Over the past decade Software Engineering research has seen a steady increase in survey-based studies, and there are several guidelines providing support for those willing to carry out surveys. The need for auditing survey research…
Context: Using student subjects in empirical studies has been discussed extensively from a methodological perspective in Software Engineering (SE), but there is a lack of similar discussion surrounding ethical aspects of doing so. As…
Among the most popular and well studied quantum characterization, verification and validation techniques is randomized benchmarking (RB), an important statistical tool used to characterize the performance of physical logic operations useful…
LLM-as-a-Judge has become a dominant approach in automated evaluation systems, playing critical roles in model alignment, leaderboard construction, quality control, and so on. However, the scalability and trustworthiness of this approach…
University research groups in Computational Science and Engineering (CSE) generally lack dedicated funding and personnel for Research Software Engineering (RSE), which, combined with the pressure to maximize the number of scientific…
Context: Expert judgement is a common method for software effort estimations in practice today. Estimators are often shown extra obsolete requirements together with the real ones to be implemented. Only one previous study has been conducted…
This note highlights the importance of investigating diversity aspects in combination in empirical research. It draws on the psychological discourse and suggests why and how software engineering scholars can use the aspect of diversity in…
User studies are paramount for advancing science. However, researchers face several barriers when performing them despite the existence of supporting tools. In this work, we study how existing tools and their features cope with previously…
Human bias is capable of changing the analysis of measured data sufficiently to alter the results of an experiment. It is incumbent upon modern experiments, especially those investigating quantities considered contentious in the broader…
Safety evaluation for advanced AI systems assumes that behavior observed under evaluation predicts behavior in deployment. This assumption weakens for agents with situational awareness, which may exploit regime leakage, cues distinguishing…
Software Engineering (SE) experiments are traditionally analyzed with statistical tests (e.g., t-tests, ANOVAs, etc.) that assume equally spread data across treatments (i.e., the homogeneity of variances assumption). Differences across…
Preference-based reinforcement learning (PbRL) has shown impressive capabilities in training agents without reward engineering. However, a notable limitation of PbRL is its dependency on substantial human feedback. This dependency stems…
Context: Requirements Engineering (RE)-related activities are critical in developing quality software and one of the most human-dependent processes in software engineering (SE). Hence, identifying the impact of diverse human-related aspects…
Understanding how software developers think, make decisions, and behave remains a key challenge in software engineering (SE). Verbalization techniques (methods that capture spoken or written thought processes) offer a lightweight and…
Background. Starting from the 1960s, practitioners and researchers have looked for ways to empirically investigate new technologies such as inspecting the effectiveness of new methods, tools, or practices. With this purpose, the empirical…
Context: As the diversity and complexity of regulations affecting Software-Intensive Products and Services (SIPS) is increasing, software engineers need to address the growing regulatory scrutiny. As with any other non-negotiable…
Conducting empirical research in software engineering industry is a process, and as such, it should be generalizable. The aim of this paper is to discuss how academic researchers may address some of the challenges they encounter during…