Related papers: Impossible Mission: Entropy Maximization with Esco…
We revisit the well-studied problem of estimating the Shannon entropy of a probability distribution, now given access to a probability-revealing conditional sampling oracle. In this model, the oracle takes as input the representation of a…
It is pointed out that the Tsallis entropy functional represented in terms of the escort distribution is not concave of the entropic index $q$ is less than unity. It is emphasized that the escort distribution is a secondary object…
We propose a generalized entropy maximization procedure, which takes into account the generalized averaging procedures and information gain definitions underlying the generalized entropies. This novel generalized procedure is then applied…
We present a near-optimal quantum algorithm, up to logarithmic factors, for estimating the Shannon entropy in the quantum probability oracle model. Our approach combines the singular value separation algorithm with quantum amplitude…
A mathematical procedure is suggested to obtain deformed entropy formulas of type K(S_K) = sum_i P_i K(-ln P_i), by requiring zero mutual K(S_K)-information between a finite subsystem and a finite reservoir. The use of this method is first…
In nonextensive statistical mechanics, two kinds of definitions have been considered for expectation valu of a physical quantity: one is the ordinary definition and the other is the normalized q-expectation value employing the escort…
We conclude a sequence of work by giving near-optimal sketching and streaming algorithms for estimating Shannon entropy in the most general streaming model, with arbitrary insertions and deletions. This improves on prior results that obtain…
Shannon entropy is the shortest average codeword length a lossless compressor can achieve by encoding i.i.d. symbols. However, there are cases in which the objective is to minimize the \textit{exponential} average codeword length, i.e. when…
Parametrized families of density operators are studied. A generalization of the lower bound of Cramer and Rao is formulated. It involves escort density operators. The notion of phi-exponential family is introduced. This family, together…
The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…
The equilibrium distributions of probabilities providing maximality of Renyi and Tsallis entropies are rederived. New S-forms of them are found which are normalised with corresponding entropies in contrast to the usual Z-forms normalised…
In this paper, we consider the information content of maximum ranked set sampling procedure with unequal samples (MRSSU) in terms of Tsallis entropy which is a nonadditive generalization of Shannon entropy. We obtain several results of…
An amended MaxEnt formulation for systems displaced from the conventional MaxEnt equilibrium is proposed. This formulation involves the minimization of the Kullback-Leibler divergence to a reference $Q$ (or maximization of Shannon…
In this paper, we investigate new procedures for statistical testing based on Tsallis entropy, a parametric generalization of Shannon entropy. Focusing on multivariate generalized Gaussian and $q$-Gaussian distributions, we develop…
A new method is proposed for analyzing complexity and studying the information in random geometric networks using Tsallis entropy tool. Tsallis entropy of the ensemble of random geometric networks is calculated based on the components of…
Regularization by the Shannon entropy enables us to efficiently and approximately solve optimal transport problems on a finite set. This paper is concerned with regularized optimal transport problems via Bregman divergence. We introduce the…
A consistent generalization of statistical mechanics is obtained by applying the maximum entropy principle to a trace-form entropy and by requiring that physically motivated mathematical properties are preserved. The emerging…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
By replacing linear averaging in Shannon entropy with Kolmogorov-Nagumo average (KN-averages) or quasilinear mean and further imposing the additivity constraint, R\'{e}nyi proposed the first formal generalization of Shannon entropy. Using…
Following a recent proof of Shannon's entropy power inequality (EPI), a comprehensive framework for deriving various EPIs for the R\'enyi entropy is presented that uses transport arguments from normal densities and a change of variable by…