Related papers: Artifact Validity in Design Science Research (DSR)…

AI-based decision-making tools are rapidly spreading across a range of real-world, complex domains like healthcare, criminal justice, and child welfare. A growing body of research has called for increased scrutiny around the validity of AI…

Peer review in software engineering research operates under tight time constraints, while generative AI has substantially reduced the human effort required to produce polished research narratives. Reviewer attention is often spent on…

Reproducibility is a cornerstone of scientific progress, yet its state in large language model (LLM)-based software engineering (SE) research remains poorly understood. This paper presents the first large-scale, empirical study of…

[Background] A well-formulated research problem is essential for achieving practical relevance in Software Engineering (SE), yet there is a lack of structured guidance in this early phase. [Aims] Our goal is to introduce and evaluate seven…

Valid empirical studies build confidence in scientific findings. Fortunately, it is now common for software engineering researchers to consider threats to validity when designing their studies and to discuss them as part of their…

Developing software for scientific applications that require the integration of diverse types of computing, instruments, and data present challenges that are distinct from commercial software. These applications require scale, and the need…

Analysis and design phases are the most crucial part of the software development life-cycle. Reusing the artifacts of these early phases is very beneficial to improve the productivity and software quality. In this paper we analyze the…

Workflow support typically focuses on single simulation experiments. This is also the case for simulation based on finite element methods. If entire simulation studies shall be supported, flexible means for intertwining revising the model,…

Large Language Models (LLMs) are increasingly embedded in academic writing practices. Although numerous studies have explored how researchers employ these tools for scientific writing, their concrete implementation, limitations, and design…

The rapid development of automated scientific survey generation technology has made it increasingly important to establish a comprehensive benchmark to evaluate the quality of generated surveys.Nearly all existing evaluation benchmarks rely…

As research increasingly relies on computational methods, the reliability of scientific results depends on the quality, reproducibility, and transparency of research software. Ensuring these qualities is critical for scientific integrity…

Training the deep neural networks that dominate NLP requires large datasets. These are often collected automatically or via crowdsourcing, and may exhibit systematic biases or annotation artifacts. By the latter we mean spurious…

Large language models (LLMs) have facilitated the generation of high-quality, cost-effective synthetic data for developing downstream models and conducting statistical analyses in various domains. However, the increased reliance on…

Retrieval-Augmented Language Models (RALMs) face significant challenges in reducing factual errors, particularly in document relevance evaluation and knowledge integration. We introduce a framework for structured relevance assessment that…

Insightful interdisciplinary collaboration is essential to the principled governance of technology. When such efforts address the interaction between computation and society, they often focus on modeling, the process by which computer…

Most currently accepted approaches to evaluating Research through Design (RtD) presume that design prototypes are finalized and ready for robust testing in laboratory or in-the-wild settings. However, it is also valuable to assess designs…

Large language models (LLMs) are rapidly being integrated into psychological research as research tools, evaluation targets, human simulators, and cognitive models. However, recent evidence reveals severe measurement unreliability:…

This paper presents the first comprehensive systematic review of literature on style-based composer identification and authorship attribution in symbolic music scores. Addressing the critical need for improved reliability and…

This paper introduces a quantitative evaluation of filters that seek to separate an image into its large-scale variations, the base layer, and its fine-scale variations, the detail layer. Such methods have proliferated with the development…

Context: The need of replicating empirical studies in Computer Science (CS) is widely recognized among the research community to consolidate acquired knowledge generalizing results. It is essential to report the changes of each replication…