Related papers: Dependence and Uncertainty: Information Measures u…
The coupled entropy, $H_\kappa,$ is proven to uniquely satisfy the requirement that a generalized entropy be a measure of the uncertainty at the scale, $\sigma,$ for a class of non-exponential distributions. The coupled stretched…
The goal of rank fusion in information retrieval (IR) is to deliver a single output list from multiple search results. Improving performance by combining the outputs of various IR systems is a challenging task. A central point is the fact…
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…
The uncertainty principle brings out intrinsic quantum bounds on the precision of measuring non-commuting observables. Statistical outcomes in the measurement of incompatible observables reveal a trade-off on the sum of corresponding…
This article proposes a space-efficient approximation to empirical tail dependence coefficients of an indefinite bivariate stream of data. The approximation, which has stream-length invariant error bounds, utilises recent work on the…
Estimating Mutual Information (MI), a key measure of dependence of random quantities without specific modelling assumptions, is a challenging problem in high dimensions. We propose a novel mutual information estimator based on parametrizing…
Causal discovery is a fundamental problem in statistics and has wide applications in different fields. Transfer Entropy (TE) is a important notion defined for measuring causality, which is essentially conditional Mutual Information (MI).…
In this work, we propose extropy measures based on density copula, distributional copula, and survival copula, and explore their properties. We study the effect of monotone transformations for the proposed measures and obtain bounds. We…
In recent years, conditional copulas, that allow dependence between variables to vary according to the values of one or more covariates, have attracted increasing attention. In high dimension, vine copulas offer greater flexibility compared…
The multivariate Hilbert-Schmidt-Independence-Criterion (dHSIC) and distance multivariance allow to measure and test independence of an arbitrary number of random vectors with arbitrary dimensions. Here we define versions which only depend…
The quantification of aleatoric and epistemic uncertainty in terms of conditional entropy and mutual information, respectively, has recently become quite common in machine learning. While the properties of these measures, which are rooted…
The formalism of statistical mechanics can be generalized by starting from more general measures of information than the Shannon entropy and maximizing those subject to suitable constraints. We discuss some of the most important examples of…
In data science, it is often required to estimate dependencies between different data sources. These dependencies are typically calculated using Pearson's correlation, distance correlation, and/or mutual information. However, none of these…
In this paper, we study entropic uncertainty relations on a finite-dimensional Hilbert space and provide several tighter bounds for multi-measurements, with some of them also valid for R\'{e}nyi and Tsallis entropies besides the Shannon…
Uncertainty relations and quantum entanglement are pivotal concepts in quantum theory. Beyond their fundamental significance in shaping our understanding of the quantum world, they also underpin crucial applications in quantum information…
We discuss the construction of component importance measures for binary coherent reliability systems from known stochastic dependence measures by measuring the dependence between system and component failures. We treat both the…
Copulas have now become ubiquitous statistical tools for describing, analysing and modelling dependence between random variables. Sklar's theorem, "the fundamental theorem of copulas", makes a clear distinction between the continuous case…
This paper introduces a comprehensive framework for Financial Information Theory by applying information-theoretic concepts such as entropy, Kullback-Leibler divergence, mutual information, normalized mutual information, and transfer…
Gibbs-Boltzmann entropy leads to systems that have a strong dependence on initial conditions. In reality, most materials behave quite independently of initial conditions. Nonextensive entropy or Tsallis entropy leads to nonextensive…
We discuss the generalized von Neumann (Tsallis) entropy and the generalized Fisher information (GFI) in nonextensive quantum systems, by using the interpolation approximation (IA) which has been shown to yield good results for the quantal…