Related papers: A Structured Evaluation Framework for Low-Code Pla…
Low-code applications are gaining popularity across various fields, enabling non-developers to participate in the software development process. However, due to the strong reliance on graphical user interfaces, they may unintentionally…
LLMs enable qualitative coding at large scale, but assessing reliability remains challenging where human experts seldom agree. We investigate confidence-diversity calibration as a quality assessment framework for accessible coding tasks…
Blockchain technology is gaining momentum across many sectors. Whereas blockchain solutions have important positive effects on the business domain, they also introduce constraints and may cause delayed or unforeseen negative effects,…
The rapid evolution of Large Language Models' has underscored the need for evaluation frameworks that are globally applicable, flexible, and modular, and that support a wide range of tasks, model types, and linguistic settings. We introduce…
Cloud computing is a disruptive technology, representing a new model for information technology (IT) solution engineering and management that promises to introduce significant cost savings and other benefits. The adoption of Cloud computing…
As AI systems advance and integrate into society, well-designed and transparent evaluations are becoming essential tools in AI governance, informing decisions by providing evidence about system capabilities and risks. Yet there remains a…
The rapid advancement of large language models (LLMs) demands increasingly reliable evaluation, yet current centralized evaluation suffers from opacity, overfitting, and hardware-induced variance. Our empirical analysis reveals an alarming…
The rapid proliferation of benchmarks has created significant challenges in reproducibility, transparency, and informed decision-making. However, unlike datasets and models -- which benefit from structured documentation frameworks like…
Stakeholders make various types of decisions with respect to requirements, design, management, and so on during the software development life cycle. Nevertheless, these decisions are typically not well documented and classified due to…
Context: While there are many success stories of achieving high reuse and improved quality using software platforms, there is a need to investigate the issues and challenges organizations face when transitioning to a software platform…
Finding a suited software solution for a company poses a resource-intensive task in an ever-widening market. Software should solve the technical task at hand as perfectly as possible and, at the same time, match the company strategy. Based…
The rapid proliferation of artificial intelligence (AI) models and methods presents growing challenges for research software engineers and researchers who must select, integrate, and maintain appropriate models within complex research…
Objective: This research explores using crowdsourcing for software usability evaluation. Background: Usability studies are essential for designing user-friendly software, but traditional methods are often costly and time-consuming.…
In modern-day organizations, many software applications require critical input to decide the next steps in the application workflow and approval. One of the most important inputs to decide the subsequent course of action is the key…
Benchmarks for large language models (LLMs) have predominantly assessed short-horizon, localized reasoning. Existing long-horizon suites (e.g. SWE-bench) rely on manually curated issues, so expanding or tuning difficulty demands expensive…
Machine Learning (ML) and Deep Learning (DL) innovations are being introduced at such a rapid pace that researchers are hard-pressed to analyze and study them. The complicated procedures for evaluating innovations, along with the lack of…
This paper addresses problems on the structural design of control systems taking explicitly into consideration the possible application to large-scale systems. We provide an efficient and unified framework to solve the following major…
Although we have seen a proliferation of algorithms for recommending visualizations, these algorithms are rarely compared with one another, making it difficult to ascertain which algorithm is best for a given visual analysis scenario.…
Modern software development teams are distributed across onsite and off-shore locations. Each team has developers with varying experience levels and English communication skills. In such a diverse development environment it is important to…
The allocation of limited resources to a large number of potential candidates presents a pervasive challenge. In the context of ranking and selecting top candidates from heteroscedastic units, conventional methods often result in…