Related papers: Enumerable Distributions, Randomness, Dependence
In this paper we prove a theorem about regression, in that the shortest description of a function consistent with a finite sample of data is less than the combined conditional Kolmogorov complexities over the data in the sample.
Measuring Mutual Information (MI) between high-dimensional, continuous, random variables from observed samples has wide theoretical and practical applications. Recent work, MINE (Belghazi et al. 2018), focused on estimating tight…
Several important measures of quantum correlations of a state of a finite-dimensional composite system are defined as linear combinations of marginal entropies of this state. This paper is devoted to the infinite-dimensional generalizations…
The mutual information (MI) between two random variables is an important correlation measure in data analysis. The Shannon entropy of a joint probability distribution is the variable part under fixed marginals. We aim to minimize and…
Recent advances in statistical learning theory have revealed profound connections between mutual information (MI) bounds, PAC-Bayesian theory, and Bayesian nonparametrics. This work introduces a novel mutual information bound for…
This paper defines a new pseudometric for binary relations between finite sets that measures consensus among subsets. The main results are (1) a concise restatement of this pseudometric with an intuitively appealing interpretation via a…
Although there is growing interest in measuring integrated information in computational and cognitive systems, current methods for doing so in practice are computationally unfeasible. Existing and novel integration measures are investigated…
Mutual Information (MI) is an useful tool for the recognition of mutual dependence berween data sets. Differen methods for the estimation of MI have been developed when both data sets are discrete or when both data sets are continuous. The…
Information-theoretic generalization bounds based on the supersample construction are a central tool for algorithm-dependent generalization analysis in the batch i.i.d.~setting. However, existing supersample conditional mutual information…
Kolmogorov's foundation of probability takes measure spaces, $\sigma$-algebras, and probability measures as basic objects. It is, however, widely recognized that this classical framework is inadequate for random phenomena involving quantum…
We introduce a notion of computable randomness for infinite sequences that generalises the classical version in two important ways. First, our definition of computable randomness is associated with imprecise probability models, in the sense…
For any $n$-partite state $\rho_{A_{1}A_{2}\cdot\cdot\cdot A_{n}}$, we define its quantum mutual information matrix as an $n$ by $n$ matrix whose $(i,j)$-entry is given by quantum mutual information $I(\rho_{A_{i}A_{j}})$. Although each…
We introduce a new information-theoretic formulation of quantum measurement uncertainty relations, based on the notion of relative entropy between measurement probabilities. In the case of a finite-dimensional system and for any approximate…
We provide finite-sample distribution approximations, that are uniform in the parameter, for inference in linear mixed models. Focus is on variances and covariances of random effects in cases where existing theory fails because their…
The accessible information quantifies the amount of classical information that can be extracted from an ensemble of quantum states. Analogously, the informational power quantifies the amount of classical information that can be extracted by…
The random matrix ensembles (RME) of quantum statistical Hamiltonian operators, e.g. Gaussian random matrix ensembles (GRME) and Ginibre random matrix ensembles (Ginibre RME), are applied to following quantum statistical systems: nuclear…
We propose a new interpretation of measures of information and disorder by connecting these concepts to group theory in a new way. Entropy and group theory are connected here by their common relation to sets of permutations. A combinatorial…
The Mutual Information (MI) is an often used measure of dependency between two random variables utilized in information theory, statistics and machine learning. Recently several MI estimators have been proposed that can achieve parametric…
Two-term asymptotic formulae for the probability distribution functions for the smallest eigenvalue of the Jacobi $ \beta $-Ensembles are derived for matrices of large size in the r\'egime where $ \beta > 0 $ is arbitrary and one of the…
Estimating mutual information (MI) is a fundamental task in data science and machine learning. Existing estimators mainly rely on either highly flexible models (e.g., neural networks), which require large amounts of data, or overly…