Related papers: Entropy Distance
Relative entropy between two states in the same Hilbert space is a fundamental statistical measure of the distance between these states. Relative entropy is always positive and increasing with the system size. Interestingly, for two states…
This work presents several expected generalization error bounds based on the Wasserstein distance. More specifically, it introduces full-dataset, single-letter, and random-subset bounds, and their analogues in the randomized subsample…
While the channel capacity reflects a theoretical upper bound on the achievable information transmission rate in the limit of infinitely many bits, it does not characterise the information transfer of a given encoding routine with finitely…
A pattern of a sequence is a sequence of integer indices with each index describing the order of first occurrence of the respective symbol in the original sequence. In a recent paper, tight general bounds on the block entropy of patterns of…
How to quantify the distance between any two partitions of a finite set is an important issue in statistical classification, whenever different clustering results need to be compared. Developing from the traditional Hamming distance between…
An $[[n,k,d]]$ quantum maximum-distance-separable code maps $k$ source qudits to $n$ coded qudits such that any $n-(d-1)$ coded qudits may recover all source qudits and $n = k + 2 (d-1)$. The entropy of the joint state of the reference…
Stirring a two-dimensional viscous fluid with rods is often an effective way to mix. The topological features of periodic rod motions give a lower bound on the topological entropy of the induced flow map, since material lines must `catch'…
We study a curve of Gibbsian families of complex 3x3-matrices and point out new features, absent in commutative finite-dimensional algebras: a discontinuous maximum-entropy inference, a discontinuous entropy distance and non-exposed faces…
Over the last few years, machine learning unlocked previously infeasible features for compression, such as providing guarantees for users' privacy or tailoring compression to specific data statistics (e.g., satellite images or audio…
We study entanglement properties of systems with spontaneously broken continuous symmetry. We find that in addition to the expected area law behavior, the entanglement entropy contains a subleading contribution which diverges…
Based on the recently introduced averaging procedure in phase space, a new type of entropy is defined on the von Neumann lattice. This quantity can be interpreted as a measure of uncertainty associated with simultaneous measurement of the…
A simple derivation of the bound on entropy is given and the holographic principle is discussed. We estimate the number of quantum states inside space region on the base of uncertainty relation. The result is compared with the Bekenstein…
In gauge theories the presence of constraints can obstruct expressing the global Hilbert space as a tensor product of the Hilbert spaces corresponding to degrees of freedom localized in complementary regions. In algebraic terms, this is due…
The finiteness of black hole entropy suggest that spacetime is fundamentally discrete, and hints at an underlying relationship between geometry and "information". The foundation of this relationship is yet to be uncovered, but should…
In this paper we seek to understand what current knowledge of entanglement entropies suggests about the appropriate way to interpret the covariant entropy bound. We first begin by arguing that just as in the classical case, a universal…
Heisenberg's uncertainty principle has recently led to general measurement uncertainty relations for quantum systems: incompatible observables can be measured jointly or in sequence only with some unavoidable approximation, which can be…
The maximum entropy that can be stored in a bounded region of space is in dispute: it goes as volume, implies (non-gravitational) microphysics; it goes as the surface area, asserts the "holographic principle." Here I show how the…
We study the finite term of the holographic entanglement entropy for the charged black hole in AdS(d+2) and other examples of black holes when the spatial region in the boundary theory is given by one or two parallel strips. For one large…
The cognitive framework of conceptual spaces [3] provides geometric means for representing knowledge. A conceptual space is a high-dimensional space whose dimensions are partitioned into so-called domains. Within each domain, the Euclidean…
In [1] it is shown that recurrent neural networks (RNNs) can learn - in a metric entropy optimal manner - discrete time, linear time-invariant (LTI) systems. This is effected by comparing the number of bits needed to encode the…