Related papers: Information Distance
Starting with a similarity function between objects, it is possible to define a distance metric on pairs of objects, and more generally on probability distributions over them. These distance metrics have a deep basis in functional analysis,…
In the past three decades, many theoretical measures of complexity have been proposed to help understand complex systems. In this work, for the first time, we place these measures on a level playing field, to explore the qualitative…
We study the complexity of approximations to the normalized information distance. We introduce a hierarchy of computable approximations by considering the number of oscillations. This is a function version of the difference hierarchy for…
A growing interest in complex networks theory results in an ongoing demand for new analytical tools. We propose a novel measure based on information theory that provides a new perspective for a better understanding of networked systems:…
Any physical system can be viewed from the perspective that information is implicitly represented in its state. However, the quantification of this information when it comes to complex networks has remained largely elusive. In this work, we…
We study an information analogue of infinitely divisible probability distributions, where the i.i.d. sum is replaced by the joint distribution of an i.i.d. sequence. A random variable $X$ is called informationally infinitely divisible if,…
According to Kolmogorov complexity, every finite binary string is compressible to a shortest code -- its information content -- from which it is effectively recoverable. We investigate the extent to which this holds for infinite binary…
Information theory establishes the fundamental limits on data transmission, storage, and processing. Quantum information theory unites information theoretic ideas with an accurate quantum-mechanical description of reality to give a more…
Distributed compression is the task of compressing correlated data by several parties, each one possessing one piece of data and acting separately. The classical Slepian-Wolf theorem (D. Slepian, J. K. Wolf, IEEE Transactions on Inf.…
The coarse similarity class $[A]$ of $A$ is the set of all $B$ whose symmetric difference with $A$ has asymptotic density 0. There is a natural metric $\delta$ on the space $\mathcal{S}$ of coarse similarity classes defined by letting…
Algorithmic statistics has two different (and almost orthogonal) motivations. From the philosophical point of view, it tries to formalize how the statistics works and why some statistical models are better than others. After this notion of…
Information theory is built on probability measures and by definition a probability measure has total mass 1. Probability measures are used to model uncertainty, and one may ask how important it is that the total mass is one. We claim that…
We study multivariate problems like function approximation, numerical integration, global optimization and dispersion. We obtain new results on the information complexity $n(\varepsilon,d)$ of these problems. The information complexity is…
Mutual information $I(X;Y)$ is a useful definition in information theory to estimate how much information the random variable $Y$ holds about the random variable $X$. One way to define the mutual information is by comparing the joint…
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome's plausibility. Information measures based on…
Real-world data typically contain a large number of features that are often heterogeneous in nature, relevance, and also units of measure. When assessing the similarity between data points, one can build various distance measures using…
We provide a derivation of quantum theory in which the existence of an energy observable that generates the reversible dynamics follows directly from information-theoretic principles. Our first principle is that every reversible dynamics is…
Quantum mechanics and information theory are among the most important scientific discoveries of the last century. Although these two areas initially developed separately it has emerged that they are in fact intimately related. In this…
We investigate fundamental connections between thermodynamics and quantum information theory. First, we show that the operational framework of thermal operations is nonequivalent to the framework of Gibbs-preserving maps, and we comment on…
Words and phrases acquire meaning from the way they are used in society, from their relative semantics to other words and phrases. For computers the equivalent of `society' is `database,' and the equivalent of `use' is `way to search the…