相关论文: KARB Solution: Compliance to Quality by Rule Based…

Benchmarking involves designing, running and disseminating rigorous performance assessments of methods, most often for data analysis and software tools, but the process can also be applied to experimental systems. Ideally, a benchmarking…

Two indicators are classically used to evaluate the quality of rule-based classification systems: predictive accuracy, i.e. the system's ability to successfully reproduce learning data and coverage, i.e. the proportion of possible cases for…

In this report we propose a solution to problem of the dependency on the experience of the software project quality assurance personnel by providing a transparent, objective and measurement based quality framework. The framework helps the…

The performance of wake-up radios must be clearly measured and understood while designing and developing robust, dependable, and affordable systems, considering both benefits and shortcomings. State-of-the-art WURs display significant…

In recent years, quantum, quantum-inspired, and hybrid algorithms are increasingly showing promise for solving software engineering optimization problems. However, best-intended practices for conducting empirical studies have not yet well…

Quantum Computing (QC) is undergoing a high rate of development, investment and research devoted to its improvement.However, there is little consensus in the industry and wider literature as to what improvement might consist of beyond…

We study the problem of auditing the fairness of a given classifier under partial feedback, where true labels are available only for positively classified individuals, (e.g., loan repayment outcomes are observed only for approved…

As Large Language Models (LLMs) advance, their potential for widespread societal impact grows simultaneously. Hence, rigorous LLM evaluations are both a technical necessity and social imperative. While numerous evaluation benchmarks have…

In its many variants, randomized benchmarking (RB) is a broadly used technique for assessing the quality of gate implementations on quantum computers. A detailed theoretical understanding and general guarantees exist for the functioning and…

We propose a system for marking sensitive or copyrighted texts to detect their use in fine-tuning large language models under black-box access with statistical guarantees. Our method builds digital ``marks'' using invisible Unicode…

The proliferation of automatic faithfulness metrics for summarization has produced a need for benchmarks to evaluate them. While existing benchmarks measure the correlation with human judgements of faithfulness on model-generated summaries,…

The advent of the Java Card standard has been a major turning point in smart card technology. With the growing acceptance of this standard, understanding the performance behavior of these platforms is becoming crucial. To meet this need, we…

Matching patients effectively and efficiently for clinical trials is a significant challenge due to the complexity and variability of patient profiles and trial criteria. This paper introduces \textbf{Multi-Agent for Knowledge Augmentation…

Software quality-in-use comprehends the quality from user's perspectives. It has gained its importance in e-learning applications, mobile service based applications and project management tools. User's decisions on software acquisitions are…

Aligning AI with human values is a pressing unsolved problem. To address the lack of quantitative metrics for value alignment, we propose EigenBench: a black-box method for comparatively benchmarking language models' values. Given an…

In this paper, the problem of load uncertainty in compliance problems is addressed where the uncertainty is described in the form of a set of finitely many loading scenarios. Computationally more efficient methods are proposed to exactly…

Clinicians often do not sufficiently adhere to evidence-based clinical guidelines in a manner sensitive to the context of each patient. It is important to detect such deviations, typically including redundant or missing actions, even when…

Language Models (LMs) have shown promising performance in natural language generation. However, as LMs often generate incorrect or hallucinated responses, it is crucial to correctly quantify their uncertainty in responding to given inputs.…

[Context and motivation] Quality requirements (QRs) are inherently diffi-cult to manage as they are often subjective, context-dependent and hard to fully grasp by various stakeholders. Furthermore, there are many sources that can provide…

Randomized benchmarking and variants thereof, which we collectively call RB+, are widely used to characterize the performance of quantum computers because they are simple, scalable, and robust to state-preparation and measurement errors.…