Related papers: Enumerable Distributions, Randomness, Dependence
A new class of distances appropriate for measuring similarity relations between sequences, say one type of similarity per distance, is studied. We propose a new ``normalized information distance'', based on the noncomputable notion of…
Distributionally robust optimization is used to tackle decision making problems under uncertainty where the distribution of the uncertain data is ambiguous. Many ambiguity sets have been proposed for continuous uncertainty that build on…
We introduce two new classes of measures of information for statistical experiments which generalise and subsume $\phi$-divergences, integral probability metrics, $\mathfrak{N}$-distances (MMD), and $(f,\Gamma)$ divergences between two or…
This work develops tools to understand how quantum information spreads, scrambles, and is reshaped by measurements in many-body systems. First, I study scrambling and pseudorandomness in the Brownian Sachdev-Ye-Kitaev (SYK) model,…
We introduce the concept of an extremely negatively dependent (END) sequence of random variables with a given common marginal distribution. The END structure, as a new benchmark for negative dependence, is comparable to comonotonicity and…
The (prefix-free) Kolmogorov complexity of a finite binary string is the length of the shortest description of the string. This gives rise to some `standard' lowness notions for reals: A is K-trivial if its initial segments have the lowest…
Estimating Mutual Information (MI), a key measure of dependence of random quantities without specific modelling assumptions, is a challenging problem in high dimensions. We propose a novel mutual information estimator based on parametrizing…
Information-Theoretical restrictions on the systems self-descriptions are applied to Quantum Measurements Theory. For the quantum object S measurement by information system O such restrictions are described by the restricted states…
Morozova and Chentsov (Morozova and Chentsov 90) studied Riemannian metrics on the set of probability measures. They showed that, up to a constant factor, the Fisher information is the only Riemannian metric which is monotone under…
The necessary information to distinguish a local inhomogeneous mass density field from its spatial average on a compact domain of the universe can be measured by relative information entropy. The Kullback-Leibler (KL) formula arises very…
Information theory is built on probability measures and by definition a probability measure has total mass 1. Probability measures are used to model uncertainty, and one may ask how important it is that the total mass is one. We claim that…
We present some new results which relate information to chaotic dynamics. In our approach the quantity of information is measured by the Algorithmic Information Content (Kolmogorov complexity) or by a sort of computable version of it…
We propose a test of independence of two multivariate random vectors, given a sample from the underlying population. Our approach, which we call MINT, is based on the estimation of mutual information, whose decomposition into joint and…
We show that the mutual information, in the sense of Kolmogorov complexity, of any pair of strings $x$ and $y$ is equal, up to logarithmic precision, to the length of the longest shared secret key that two parties, one having $x$ and the…
Multivariate mutual information provides a conceptual framework for characterizing higher-order interactions in complex systems. Two well-known measures of multivariate information---total correlation and dual total correlation---admit a…
The purpose of this paper is to answer two questions left open in [B. Durand, A. Shen, and N. Vereshchagin, Descriptive Complexity of Computable Sequences, Theoretical Computer Science 171 (2001), pp. 47--58]. Namely, we consider the…
We try to establish a unified information theoretic approach to learning and to explore some of its applications. First, we define {\em predictive information} as the mutual information between the past and the future of a time series,…
We study a proper definition of R\'enyi mutual information (RMI) in quantum field theory as defined via the Petz R\'enyi relative entropy. Unlike the standard definition, the RMI we compute is a genuine measure of correlations between…
We define Persistent Mutual Information (PMI) as the Mutual (Shannon) Information between the past history of a system and its evolution significantly later in the future. This quantifies how much past observations enable long term…
The data for many classification problems, such as pattern and speech recognition, follow mixture distributions. To quantify the optimum performance for classification tasks, the Shannon mutual information is a natural information-theoretic…