Related papers: Information theory with finite vector spaces
Shannon entropy and Fisher information functionals are known to quantify certain information-theoretic properties of continuous probability distributions of various origins. We carry out a systematic study of these functionals, while…
We present an information-theoretic analysis of the Cosmic Web that goes beyond the scalar density contrast and exploits the full structure of the tidal deformation tensor. The three eigenvalues ($\lambda_1, \lambda_2, \lambda_3$) of the…
For statistical systems that violate one of the four Shannon-Khinchin axioms, entropy takes a more general form than the Boltzmann-Gibbs entropy. The framework of superstatistics allows one to formulate a maximum entropy principle with…
From the output produced by a memoryless deletion channel with a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…
Power-law distributions are widely observed in complex systems, yet establishing their thermodynamic consistency remains a theoretical challenge. In this paper, we present a thermodynamic framework for power-law statistics based on the…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
Superstatistics is a generalization of equilibrium statistical mechanics that describes systems in nonequilibrium steady states. Among the possible superstatistical distributions, the $q$-canonical ensemble (also known as Tsallis'…
This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error…
Generalizing the group structure of the Euclidean space, we construct a Riemannian metric on the deformed set \ $\mathbb{R}^n_q$ \ induced by the Tsallis entropy composition property. We show that the Tsallis entropy is a "hyperbolic…
This paper derives bounds for two omnipresent information theoretic measures, the Shannon entropy and its complementary dual, the extropy. Based on a large size data set from a logconcave model, the said bounds are obtained for the entropy…
We discuss the generalized von Neumann (Tsallis) entropy and the generalized Fisher information (GFI) in nonextensive quantum systems, by using the interpolation approximation (IA) which has been shown to yield good results for the quantal…
We examine the minimization of information entropy for measures on the phase space of bounded domains, subject to constraints that are averages of grand canonical distributions. We describe the set of all such constraints and show that it…
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…
Based on the form invariance of the structures given by Khinchin's axiomatic foundations of information theory and the pseudoadditivity of the Tsallis entropy indexed by q, the concept of conditional entropy is generalized to the case of…
The Tsallis entropy given for a positive parameter $\alpha$ can be considered as a modification of the classical Shannon entropy. For the latter, corresponding to $\alpha=1$, there exist many axiomatic characterizations. One of them based…
We discuss the dependence of the Shannon entropy of normalized finite rank-1 POVMs on the choice of the input state, looking for the states that minimize this quantity. To distinguish the class of measurements where the problem can be…
We examine the inference of quantum density operators from incomplete information by means of the maximization of general non-additive entropic forms. Extended thermodynamic relations are given. When applied to a bipartite spin 1/2 system,…
This paper considers the estimation of Shannon entropy for discrete distributions with countably infinite support. While minimax rates for finite-support distributions are established, infinite-support distributions present distinct…
The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…
We present an embedding of the Tsallis entropy into the 3-dimensional Heisenberg group, in order to understand the meaning of generalized independence as encoded in the Tsallis entropy composition property. We infer that the Tsallis entropy…