Related papers: Expert Elicitation for Reliable System Design
As we develop more assistive and automated game design systems, the question of how these systems should be integrated into game development workflows, and how much adaptation may be required, becomes increasingly important. In this paper…
Critical systems, such as those used in healthcare, defence, and disaster management, demand rigorous requirements engineering to ensure safety and reliability. Yet, much of this rigour has traditionally focused on technical assurance,…
Importance of addressing soft errors in both safety critical applications and commercial consumer products is increasing, mainly due to ever shrinking geometries, higher-density circuits, and employment of power-saving techniques such as…
As large language models (LLMs) continue to advance, reliable evaluation methods are essential particularly for open-ended, instruction-following tasks. LLM-as-a-Judge enables automatic evaluation using LLMs as evaluators, but its…
Future energy systems are subject to various uncertain influences. As resilient systems they should maintain a constantly high operational performance whatever happens. We explore different levels and time scales of decision making in…
Crisis management is a complex problem raised by the scientific community currently. Decision support systems are a suitable solution for such issues, they are indeed able to help emergency managers to prevent and to manage crisis in…
Subjective judgements from experts provide essential information when assessing and modelling threats in respect to cyber-physical systems. For example, the vulnerability of individual system components can be described using multiple…
Obviously, the dynamism of software reliability research has speeded up significantly in the last period, and we can state the fact that its intensity is approaching, and in some cases is ahead of the information systems hardware…
Commercial Cloud services have been increasingly supplied to customers in industry. To facilitate customers' decision makings like cost-benefit analysis or Cloud provider selection, evaluation of those Cloud services are becoming more and…
A primary goal of science and engineering (S & E) education is to produce good problem solvers, but how to best teach and measure the quality of problem-solving remains unclear. The process is complex, multifaceted, and not fully…
Score reliability is necessary for establishing a validity argument for an instrument, and is therefore highly important to investigate. Depending on the proposed instrument use and score interpretations, differing degrees of precision in…
At the initial design stage engineers often rely on low-fidelity models that have high epistemic uncertainty. Traditional safety-margin-based deterministic design resorts to testing (e.g. prototype experiment, evaluation of high-fidelity…
Reliable effort estimation remains an ongoing challenge to software engineers. Accurate effort estimation is the state of art of software engineering, effort estimation of software is the preliminary phase between the client and the…
We draw on our experience working on system and software assurance and evaluation for systems important to society to summarise how safety engineering is performed in traditional critical systems, such as aircraft flight control. We analyse…
Explainability is one of the key ethical concepts in the design of AI systems. However, attempts to operationalize this concept thus far have tended to focus on approaches such as new software for model interpretability or guidelines with…
We consider the problem of eliciting expert assessments of an uncertain parameter. The context is risk control, where there are, in fact, three uncertain parameters to be estimates. Two of these are probabilities, requiring the that the…
Recently, requirements for the explainability of software systems have gained prominence. One of the primary motivators for such requirements is that explainability is expected to facilitate stakeholders' trust in a system. Although this…
Many development decisions affect the results obtained from ML experiments: training data, features, model architecture, hyperparameters, test data, etc. Among these aspects, arguably the most important design decisions are those that…
Empirical research plays a fundamental role in the machine learning domain. At the heart of impactful empirical research lies the development of clear research hypotheses, which then shape the design of experiments. The execution of…
This paper describes a normative system design that incorporates diagnosis, dynamic evolution, decision making, and information gathering. A single influence diagram demonstrates the design's coherence, yet each activity is more effectively…