Related papers: A simple sketching algorithm for entropy estimatio…
Relations between Shannon entropy and Renyi entropies of integer order are discussed. For any N-point discrete probability distribution for which the Renyi entropies of order two and three are known, we provide an lower and an upper bound…
This paper proposes $\alpha$-GAN, a generative adversarial network using R\'{e}nyi measures. The value function is formulated, by R\'{e}nyi cross entropy, as an expected certainty measure incurred by the discriminator's soft decision as to…
For studies in reliability, biometry, and survival analysis, the length-biased distribution is often well-suited for certain natural sampling plans. In this paper, we study the strong uniform consistency of two nonparametric estimators for…
Following a recent proof of Shannon's entropy power inequality (EPI), a comprehensive framework for deriving various EPIs for the R\'enyi entropy is presented that uses transport arguments from normal densities and a change of variable by…
We study the problem of discovering the simplest latent variable that can make two observed discrete variables conditionally independent. The minimum entropy required for such a latent is known as common entropy in information theory. We…
We report multipronged progress on the stochastic averaging approach to numerical analytic continuation of quantum Monte Carlo data. With the sampled spectrum parametrized with delta-functions in continuous frequency space, a calculation of…
In this paper, we consider the problem of variable-length source coding allowing errors. The exponential moment of the codeword length is analyzed in the non-asymptotic regime and in the asymptotic regime. Our results show that the smooth…
The statistical analysis of data stemming from dynamical systems, including, but not limited to, time series, routinely relies on the estimation of information theoretical quantities, most notably Shannon entropy. To this purpose, possibly…
The behavior of the Kozachenko - Leonenko estimates for the (differential) Shannon entropy is studied when the number of i.i.d. vector-valued observations tends to infinity. The asymptotic unbiasedness and L^2-consistency of the estimates…
We develop a maximum relative entropy formalism to generate optimal approximations to probability distributions. The central results consist in (a) justifying the use of relative entropy as the uniquely natural criterion to select a…
Shannon-Renyi and stabilizer entropies are key diagnostics of structure, non-stabilizerness, phase transitions, and universality in quantum many-body states. We establish an exact correspondence for quadratic fermions: for any nondegenerate…
The Renyi entropy plays an essential role in quantum information theory. We study the continuity estimation of the Renyi entropy. An inequality relating the Renyi entropy difference of two quantum states to their trace norm distance is…
Analytic continuation of numerical data obtained in imaginary time or frequency has become an essential part of many branches of quantum computational physics. It is, however, an ill-conditioned procedure and thus a hard numerical problem.…
Many statistical estimators are defined as the fixed point of a data-dependent operator, with estimators based on minimizing a cost function being an important special case. The limiting performance of such estimators depends on the…
Many partially-successful attempts have been made to find the most natural discrete-variable version of Shannon's entropy power inequality (EPI). We develop an axiomatic framework from which we deduce the natural form of a discrete-variable…
We study parameter estimation and asymptotic inference for sparse nonlinear regression. More specifically, we assume the data are given by $y = f( x^\top \beta^* ) + \epsilon$, where $f$ is nonlinear. To recover $\beta^*$, we propose an…
The stretched exponential function, $\exp[-(t/\tau_{K})^{\beta}]$, describes various relaxation processes while it has been suggested that the power exponent, $\beta$ is derived from the non-uniformity of the process. In this paper, we…
A Massey-like inequality is any useful lower bound on guessing entropy in terms of the computationally scalable Shannon entropy. The asymptotically optimal Massey-like inequality is determined and further refined for finite-support…
Regularization by the Shannon entropy enables us to efficiently and approximately solve optimal transport problems on a finite set. This paper is concerned with regularized optimal transport problems via Bregman divergence. We introduce the…
Measuring the predictability and complexity of time series using entropy is essential tool de-signing and controlling a nonlinear system. However, the existing methods have some drawbacks related to the strong dependence of entropy on the…