Related papers: Information Spectrum Converse for Minimum Entropy …
Numerous entropy-type characteristics (functionals) generalizing R\'enyi entropy are widely used in mathematical statistics, physics, information theory, and signal processing for characterizing uncertainty in probability distributions and…
Information theory, though originally developed for communications engineering, provides mathematical tools with broad applications across science. These tools characterize the fundamental limits of data compression and transmission in the…
We show how convergence to the Gumbel distribution in an extreme value setting can be understood in an information-theoretic sense. We introduce a new type of score function which behaves well under the maximum operation, and which implies…
For several styles of fidelity constraints -- guaranteed distortion, conditional excess distortion, excess distortion -- we show mutual information upper bounds on the minimum expected description length needed to represent a random…
Functional data that are nonnegative and have a constrained integral can be considered as samples of one-dimensional density functions. Such data are ubiquitous. Due to the inherent constraints, densities do not live in a vector space and,…
Entropic causal inference is a recent framework for learning the causal graph between two variables from observational data by finding the information-theoretically simplest structural explanation of the data, i.e., the model with smallest…
The operational characterization of quantum coherence is the corner stone in the development of resource theory of coherence. We introduce a new coherence quantifier based on max-relative entropy. We prove that max-relative entropy of…
We consider some formulations of the entropy bounds at the semiclassical level. The entropy S(V) localized in a region V is divergent in quantum field theory (QFT). Instead of it we focus on the mutual information I(V,W)=S(V)+S(W)-S(V\cup…
The entanglement spectrum, i.e., the full distribution of Schmidt eigenvalues of the reduced density matrix, contains more information than the conventional entanglement entropy and has been studied recently in several many-particle…
In vector quantization the number of vectors used to construct the codebook is always an undefined problem, there is always a compromise between the number of vectors and the quantity of information lost during the compression. In this text…
This paper considers the problem of sequential empirical coordination, where the objective is to achieve a given value of the expected uniform deviation between state-action empirical averages and statistical expectations under a given…
The inverse problem of statistical mechanics involves finding the minimal Hamiltonian that is consistent with some observed set of correlation functions. This problem has received renewed interest in the analysis of biological networks; in…
This work is an enquiry into the circumstances under which entropy methods can give an answer to the questions of both quantum separability and classical correlations of a composite state. Several entropy functionals are employed to examine…
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
The problem of finding a reduced dimensionality representation of categorical variables while preserving their most relevant characteristics is fundamental for the analysis of complex data. Specifically, given a co-occurrence matrix of two…
In statistical physics entropy is usually introduced as a global quantity which expresses the amount of information that would be needed to specify the microscopic configuration of a system. However, for lattice models with infinitely many…
We establish a lower bound on the entropy of weighted sums of (possibly dependent) random variables $(X_1, X_2, \dots, X_n)$ possessing a symmetric joint distribution. Our lower bound is in terms of the joint entropy of $(X_1, X_2, \dots,…
Lower bounds for the average probability of error of estimating a hidden variable X given an observation of a correlated random variable Y, and Fano's inequality in particular, play a central role in information theory. In this paper, we…
We study a stochastic control system, described by Ito controllable equation, and evaluate the solutions by an entropy functional (EF), defined by the equation functions of controllable drift and diffusion. Considering a control problem for…
We discuss the dependence of the Shannon entropy of normalized finite rank-1 POVMs on the choice of the input state, looking for the states that minimize this quantity. To distinguish the class of measurements where the problem can be…