Related papers: Testing the Unknown: A Framework for OpenMP Testin…
Mutation testing is a standard technique to evaluate the quality of a test suite. Due to its computationally intensive nature, many approaches have been proposed to make this technique feasible in real case scenarios. Among these…
Smart contracts are computer programs that run on blockchain platforms, with Solidity being the most widely used language for their development. As blockchain technology advances, smart contracts have become increasingly important across…
Currently, statistical tests for random number generators (RNGs) are widely used in practice, and some of them are even included in information security standards. But despite the popularity of RNGs, consistent tests are known only for…
The recent surge of building software systems powered by Large Language Models (LLMs) has led to the development of various testing frameworks, primarily focused on treating prompt templates as the unit of testing. Despite the significant…
Boolean formulae compactly encode huge, constrained search spaces. Thus, variability-intensive systems are often encoded with Boolean formulae. The search space of a variability-intensive system is usually too large to explore without…
The prevalence of software systems has become an integral part of modern-day living. Software usage has increased significantly, leading to its growth in both size and complexity. Consequently, software development is becoming a more…
Unfair behaviors of Machine Learning (ML) software have garnered increasing attention and concern among software engineers. To tackle this issue, extensive research has been dedicated to conducting fairness testing of ML software, and this…
Statistical evaluation aims to estimate the generalization performance of a model using held-out i.i.d.\ test data sampled from the ground-truth distribution. In supervised learning settings such as classification, performance metrics such…
The use of large language models for code generation is a rapidly growing trend in software development. However, without effective methods for ensuring the correctness of generated code, this trend could lead to undesirable outcomes. In…
In the uniformity testing task, an algorithm is provided with samples from an unknown probability distribution over a (known) finite domain, and must decide whether it is the uniform distribution, or, alternatively, if its total variation…
In recent months, large language models (LLMs) have made significant progress in mathematical proof generation, but further advancement is hindered by the lack of a large-scale, high-quality dataset of human-evaluated proofs. While…
Quantum technologies provide many applications for information processing tasks that are impossible to realize within classical physics. These capabilities include such fundamental resources as generating secure, i.e. private and…
Random numbers are indispensable for a variety of applications ranging from testing physics foundation to information encryption. In particular, nonlocality tests provide a strong evidence to our current understanding of nature -- quantum…
Generating fake data is an essential dimension of modern software testing, as demonstrated by the number and significance of data faking libraries. Yet, developers of faking libraries cannot keep up with the wide range of data to be…
The ultimate random number generators are those certified to be unpredictable -- including to an adversary. The use of simple quantum processes promises to provide numbers that no physical observer could predict but, in practice, unwanted…
We have introduced in former work the concept of Deep Randomness and its interest to design Unconditionally Secure communication protocols. We have in particular given an example of such protocol and introduced how to design a Deep Random…
To evaluate a classification algorithm, it is common practice to plot the ROC curve using test data. However, the inherent randomness in the test data can undermine our confidence in the conclusions drawn from the ROC curve, necessitating…
We consider a zero-error probabilistic group testing problem where individuals are defective independently but not with identical probabilities. We propose a greedy set formation method to build sets of individuals to be tested together. We…
In general, randomness tests included in a test suite are not independent of each other. This renders it difficult to fix a rational criterion through the whole test suite with an explicit significance level. In this paper, we focus on the…
Given the complexity of combinations of tasks, languages, and domains in natural language processing (NLP) research, it is computationally prohibitive to exhaustively test newly proposed models on each possible experimental setting. In this…