Related papers: SimLab 1.1, Software for Sensitivity and Uncertain…

We calibrate a Natural History Model, which is a class of computer simulator used in the health industry, and here has been used to characterise bowel cancer incidence for the UK. The simulator tracks the development of bowel cancer in a…

In this paper, we study the problem of uncertainty estimation and calibration for LLMs. We begin by formulating the uncertainty estimation problem, a relevant yet underexplored area in existing literature. We then propose a supervised…

In many real-world classification tasks, label noise is an unavoidable issue that adversely affects the generalization error of machine learning models. Additionally, evaluating how methods handle such noise is complicated, as the effect…

We introduce CausaLab, a scalable environment for evaluating interactive causal discovery by LLM agents. Unlike prior evaluations, CausaLab evaluates both whether an agent can solve a problem using causal evidence and whether its answer is…

We present general principles for the design and analysis of unbiased Monte Carlo estimators in a wide range of settings. Our estimators posses finite work-normalized variance under mild regularity conditions. We apply our estimators to…

Effective human-machine collaboration requires machine learning models to externalize uncertainty, so users can reflect and intervene when necessary. For language models, these representations of uncertainty may be impacted by sycophancy…

The aim of this paper is to describe a new an integrated methodology for project control under uncertainty. This proposal is based on Earned Value Methodology and risk analysis and presents several refinements to previous methodologies.…

Background and Objective: Computational ultrasound imaging has become a well-established methodology in the ultrasound community. Simulations of ultrasound sequences and images allow the study of innovative techniques in terms of emission…

Virtual experiments (VEs), a modern tool in metrology, can be used to help perform an uncertainty evaluation for the measurand. Current guidelines in metrology do not cover the many possibilities to incorporate VEs into an uncertainty…

Simulation-based testing is a promising approach to significantly reduce the validation effort of automated driving functions. Realistic models of environment perception sensors such as camera, radar and lidar play a key role in this…

Global sensitivity analysis is used to quantify the influence of uncertain input parameters on the response variability of a numerical model. The common quantitative methods are applicable to computer codes with scalar input variables. This…

A recent research trend has emerged to identify developers' emotions, by applying sentiment analysis to the content of communication traces left in collaborative development environments. Trying to overcome the limitations posed by using…

The assessment of the reliability of Monte Carlo simulations is discussed, with emphasis on uncertainty quantification and the related impact on experimental results. Methods and techniques to account for epistemic uncertainties, i.e. for…

We present a Monte Carlo simulation framework for analysing the risk involved in deploying real-time control systems in safety-critical applications, as well as an algorithm design technique allowing one (in certain situations) to robustify…

Sentiment analysis gets increasing attention in software engineering with new tools emerging from new insights provided by researchers. Existing use cases and tools are meant to be used for textual communication such as comments on…

We present a common framework for Bayesian emulation methodologies for multivariate-output simulators, or computer models, that employ either parametric linear models or nonparametric Gaussian processes. Novel diagnostics suitable for…

In the SysLab project we develop a software engineering method based on a mathematical foundation. The SysLab system model serves as an abstract mathematical model for information systems and their components. It is used to formalize the…

As instruction-tuned large language models (LLMs) evolve, aligning pretrained foundation models presents increasing challenges. Existing alignment strategies, which typically leverage diverse and high-quality data sources, often overlook…

When conducting user studies to ascertain the usefulness of model explanations in aiding human decision-making, it is important to use real-world use cases, data, and users. However, this process can be resource-intensive, allowing only a…

Uncertainty quantification in automated image analysis is highly desired in many applications. Typically, machine learning models in classification or segmentation are only developed to provide binary answers; however, quantifying the…