Related papers: Methodological Problems in Every Black-Box Study o…

As software becomes more complex and assumes an even greater role in our lives, formal verification is set to become the gold standard in securing software systems into the future, since it can guarantee the absence of errors and entire…

Context: Ensuring safety for any sophisticated system is getting more complex due to the rising number of features and functionalities. This calls for formal methods to entrust confidence in such systems. Nevertheless, using formal methods…

Today, machine learning (ML) models are increasingly applied in decision making. This induces an urgent need for quality assurance of ML models with respect to (often domain-dependent) requirements. Monotonicity is one such requirement. It…

Firing a gun leaves marks on cartridge cases which purportedly uniquely identify the gun. Firearms examiners typically use a visual examination to evaluate if two cartridge cases were fired from the same gun, and this is a subjective…

Software reliability analysis is performed at various stages during the process of engineering software as an attempt to evaluate if the software reliability requirements have been (or might be) met. In this report, I present a summary of…

Computational science relies on scientific software as its primary instrument for scientific discovery. Therefore, similar to the use of other types of scientific instruments, correct software and the correct operation of the software is…

Assessing the validity of user simulators when used for the evaluation of information retrieval systems remains an open question, constraining their effective use and the reliability of simulation-based results. To address this issue, we…

Student experiences and empirical studies report that "black box" AI text detectors produce high false positive rates with disproportionate errors against certain student populations, yet typically theoretical analyses model detection as a…

Preparation and measurement of physical systems are the operational building blocks of any physical experiment, and to describe them is the first purpose of any physical theory. It is remarkable that, in some situations, even when only…

In the industries that involved either chemistry or biology, such as pharmaceutical industries, chemical industries or food industry, the analytical methods are the necessary eyes and hear of all the material produced or used. If the…

A system is said to be fail-secure, sometimes confused with fail-safe, if it maintains its security requirements even in the event of some faults. Fail-secure analyses are required by some validation schemes, such as some Common Criteria or…

The field of image composition is constantly trying to improve the ways in which an image can be altered and enhanced. While this is usually done in the name of aesthetics and practicality, it also provides tools that can be used to…

Deep metric learning papers from the past four years have consistently claimed great advances in accuracy, often more than doubling the performance of decade-old methods. In this paper, we take a closer look at the field to see if this is…

In this paper we examine the claims made by Bullock et. al. on the applicability of black-box injury risk approaches in the sports injury domain. Overall, we agree that transparency is necessary for Machine Learning models to be useful in…

To promote economic stability, finance should be studied as a hard science, where scientific methods apply. When a trading strategy is proposed, the underlying model should be transparent and defined robustly to allow other researchers to…

As reinforcement learning (RL) achieves more success in solving complex tasks, more care is needed to ensure that RL research is reproducible and that algorithms herein can be compared easily and fairly with minimal bias. RL results are,…

Background. There are some publications in software engineering research that aim at guiding researchers in assessing validity threats to their studies. Still, many researchers fail to address many aspects of validity that are essential to…

We demonstrate a validity problem of machine learning in the vital application area of disease diagnosis in medicine. It arises when target labels in training data are determined by an indirect measurement, and the fundamental measurements…

In the last years many accurate decision support systems have been constructed as black boxes, that is as systems that hide their internal logic to the user. This lack of explanation constitutes both a practical and an ethical issue. The…

Model checking and testing are two areas with a similar goal: to verify that a system satisfies a property. They start with different hypothesis on the systems and develop many techniques with different notions of approximation, when an…