Related papers: Generalized Maximum Entropy Methods as Limits of t…
By replacing linear averaging in Shannon entropy with Kolmogorov-Nagumo average (KN-averages) or quasilinear mean and further imposing the additivity constraint, R\'{e}nyi proposed the first formal generalization of Shannon entropy. Using…
Estimation of Shannon and R\'enyi entropies of unknown discrete distributions is a fundamental problem in statistical property testing and an active research topic in both theoretical computer science and information theory. Tight bounds on…
Maximum entropy method is a constructive criterion for setting up a probability distribution maximally non-committal to missing information on the basis of partial knowledge, usually stated as constrains on expectation values of some…
The maximum entropy principle is often used for bi-level or multi-level thresholding of images. For this purpose, some methods are available based on Shannon and Tsallis entropies. In this paper, we discuss them and propose a method based…
R\'enyi entropy is a one-parameter generalization of Shannon entropy, which has been used in various fields of physics. Despite its wide applicability, the physical interpretations of the R\'enyi entropy are not widely known. In this paper,…
This paper gives improved R\'{e}nyi entropy power inequalities (R-EPIs). Consider a sum $S_n = \sum_{k=1}^n X_k$ of $n$ independent continuous random vectors taking values on $\mathbb{R}^d$, and let $\alpha \in [1, \infty]$. An R-EPI…
In this paper, we consider the information content of maximum ranked set sampling procedure with unequal samples (MRSSU) in terms of Tsallis entropy which is a nonadditive generalization of Shannon entropy. We obtain several results of…
We develop the method of Maximum Entropy (ME) as a technique to generate approximations to probability distributions. The central results consist in (a) justifying the use of relative entropy as the uniquely natural criterion to select a…
The minimum error entropy (MEE) criterion has been successfully used in fields such as parameter estimation, system identification and the supervised machine learning. There is in general no explicit expression for the optimal MEE estimate…
We present a technique for entropy optimization to calculate a distribution from its moments. The technique is based upon maximizing a discretized form of the Shannon entropy functional by mapping the problem onto a dual space where an…
We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, R\'enyi, generalized R\'enyi, Tsallis, Sharma-Mittal, and modified Shannon entropy,…
Quantum key distribution requires tight and reliable bounds on the secret key rate to ensure robust security. This is particularly so for the regime of finite block sizes, where the optimization of generalized R\'enyi entropic quantities is…
We review a new form of entropy suggested by us, with origin in mixing of states of systems due to interactions and deformations of phase cells. It is demonstrated that this nonextensive form also leads to asymmetric maximal entropy…
Existing polarization theories have mostly been concerned with Shannon's information measures, such as Shannon entropy and mutual information, and some related measures such as the Bhattacharyya parameter. In this work, we extend…
Following a recent proof of Shannon's entropy power inequality (EPI), a comprehensive framework for deriving various EPIs for the R\'enyi entropy is presented that uses transport arguments from normal densities and a change of variable by…
We present a new class of estimators of Shannon entropy for severely undersampled discrete distributions. It is based on a generalization of an estimator proposed by T. Schuermann, which itself is a generalization of an estimator proposed…
The formalism of statistical mechanics can be generalized by starting from more general measures of information than the Shannon entropy and maximizing those subject to suitable constraints. We discuss some of the most important examples of…
Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable…
We report on an improvement to the implementation of the Maximum Entropy Method (MEM). It amounts to departing from the search space obtained through a singular value decomposition (SVD) of the Kernel. Based on the shape of the SVD basis…
We report multipronged progress on the stochastic averaging approach to numerical analytic continuation of quantum Monte Carlo data. With the sampled spectrum parametrized with delta-functions in continuous frequency space, a calculation of…