Related papers: Measuring Inconsistency in Probabilistic Knowledge…
Quantifying the inconsistency of a database is motivated by various goals including reliability estimation for new datasets and progress indication in data cleaning. Another goal is to attribute to individual tuples a level of…
In this paper, building on work done on measuring inconsistency in knowledge bases, we introduce inconsistency measures for databases. In particular, focusing on databases with denial constraints, we first consider the natural approach of…
Researchers in explainable artificial intelligence have developed numerous methods for helping users understand the predictions of complex supervised learning models. By contrast, explaining the $\textit{uncertainty}$ of model outputs has…
We investigate the problem of inconsistency measurement on large knowledge bases by considering stream-based inconsistency measurement, i.e., we investigate inconsistency measures that cannot consider a knowledge base as a whole but process…
We present machine-learning-based approaches for determining the \emph{degree} of inconsistency -- which is a numerical value -- for propositional logic knowledge bases. Specifically, we present regression- and neural-based models that…
In question-answering tasks, determining when to trust the outputs is crucial to the alignment of large language models (LLMs). Kuhn et al. (2023) introduces semantic entropy as a measure of uncertainty, by incorporating linguistic…
This paper introduces a measure of uncertainty in the determination of the Shapley value, illustrates it with examples, and studies some of its properties. The introduced measure of uncertainty quantifies random variations in a player's…
We investigate the application of inconsistency measures to the problem of analysing business rule bases. Due to some intricacies of the domain of business rule bases, a straightforward application is not feasible. We therefore develop some…
How should we quantify the inconsistency of a database that violates integrity constraints? Proper measures are important for various tasks, such as progress indication and action prioritization in cleaning systems, and reliability…
The Shapley value provides a principled framework for fairly distributing rewards among participants according to their individual contributions. While prior work has applied this concept to data valuation in machine learning, existing…
To obtain reliable results of expertise, which usually use individual and group expert pairwise comparisons, it is important to summarize (aggregate) expert estimates provided that they are sufficiently consistent. There are several ways to…
Shapley values originated in cooperative game theory but are extensively used today as a model-agnostic explanation framework to explain predictions made by complex machine learning models in the industry and academia. There are several…
There have been a number of developments in measuring inconsistency in logic-based representations of knowledge. In contrast, the development of inconsistency measures for computational models of argument has been limited. To address this…
In this report, we investigate (element-based) inconsistency measures for multisets of business rule bases. Currently, related works allow to assess individual rule bases, however, as companies might encounter thousands of such instances…
The reliability of large language models (LLMs) is greatly compromised by their tendency to hallucinate, underscoring the need for precise identification of knowledge gaps within LLMs. Various methods for probing such gaps exist, ranging…
Measuring inconsistency is viewed as an important issue related to handling inconsistencies. Good measures are supposed to satisfy a set of rational properties. However, defining sound properties is sometimes problematic. In this paper, we…
Multimodal learning combines information from multiple data modalities to improve predictive performance. However, modalities often contribute unequally and in a data dependent way, making it unclear which data modalities are genuinely…
Measures of discordance between datasets have become an essential part of cosmological analyses. It is important to accurately evaluate the significance of such discordances when present. We propose here a Bayesian interpretation of…
Pairwise comparisons are a well-known method for the representation of the subjective preferences of a decision maker. Evaluating their inconsistency has been a widely studied and discussed topic and several indices have been proposed in…
Shapley values underlie one of the most popular model-agnostic methods within explainable artificial intelligence. These values are designed to attribute the difference between a model's prediction and an average baseline to the different…