Related papers: Entropy Distance
Entropy coding is the backbone data compression. Novel machine-learning based compression methods often use a new entropy coder called Asymmetric Numeral Systems (ANS) [Duda et al., 2015], which provides very close to optimal bitrates and…
Dependence among marginally constrained observations can break a finite-sample barrier. To formalize this phenomenon, we introduce the \emph{minimum list entropy coupling} $H(P\|Q_1,\dots,Q_m)$, the minimum conditional entropy…
We present an algorithm that, given a channel, determines if there is a distance for it such that the maximum likelihood decoder coincides with the minimum distance decoder. We also show that any metric, up to a decoding equivalence, can be…
The $\lambda$-core vertices of a graph correspond to the non-zero entries of some eigenvector of $\lambda$ for a universal adjacency matrix $\mathbf{U}$ of the graph. We define a partition of the vertex set $V$ based on the $\lambda$-core…
When a spacetime has boundaries, the entangling surface does not have to be necessarily compact and it may have boundaries as well. Then there appear a new, boundary, contribution to the entanglement entropy due to the intersection of the…
We consider the entanglement entropy for a free $U(1)$ theory in $3 + 1$ dimensions in the extended Hilbert space definition. By taking the continuum limit carefully we obtain a replica trick path integral which calculates this entanglement…
It has been widely observed that there exists a fundamental trade-off between the minimum (Hamming) distance properties and the iterative decoding convergence behavior of turbo-like codes. While capacity achieving code ensembles typically…
Herding is a deterministic algorithm used to generate data points that can be regarded as random samples satisfying input moment conditions. The algorithm is based on the complex behavior of a high-dimensional dynamical system and is…
Entropy arises in strong interactions by a dynamical separation of ``partons'' from unobservable ``environment'' modes due to confinement. For interacting scalar fields we calculate the statistical entropy of the observable subsystem.…
The metric properties of the set in which random variables take their values lead to relevant probabilistic concepts. For example, the mean of a random variable is a best predictor in that it minimizes the standard Euclidean distance or…
We introduce a novel notion of invariance feedback entropy to quantify the state information that is required by any controller that enforces a given subset of the state space to be invariant. We establish a number of elementary properties,…
This article presents the calculation of the entropy of a system with Zipfian distribution and shows that a communication system tends to present an exponent value close to one, but still greater than one, so that it might maximize entropy…
We analyze complexity in spatial network ensembles through the lens of graph entropy. Mathematically, we model a spatial network as a soft random geometric graph, i.e., a graph with two sources of randomness, namely nodes located randomly…
This paper presents methods to compare networks where relationships between pairs of nodes in a given network are defined. We define such network distance by searching for the optimal method to embed one network into another network, prove…
An embedding of a code is a mapping that preserves distances between codewords. We prove that any code with code distance $\rho$ and length $d$ can be embedded into an MDS code with the same code distance and length but under a larger…
The scattering cross section is the effective area of collision when two particles collide. Quantum mechanically, it is a measure of the probability for a specific process to take place. Employing wave packets to describe the scattering…
Given a factor code $\pi$ from a one-dimensional shift of finite type $X$ onto an irreducible sofic shift $Y$, if $\pi$ is finite-to-one there is an invariant called the degree of $\pi$ which is defined the number of preimages of a typical…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
We study entanglement entropy for regions with a singular boundary in higher dimensions using the AdS/CFT correspondence and find that various singularities make new universal contributions. When the boundary CFT has an even spacetime…
A scalar field in the ground state, when partially hidden from observation by a spherical boundary, acquires entanglement entropy $S$ proportional to the area of the surface. This area law is well established in flat space, where it follows…