Related papers: Generalized Maximum Entropy Methods as Limits of t…
Many partially-successful attempts have been made to find the most natural discrete-variable version of Shannon's entropy power inequality (EPI). We develop an axiomatic framework from which we deduce the natural form of a discrete-variable…
We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction…
Two R\'{e}nyi-type generalizations of the Shannon cross-entropy, the R\'{e}nyi cross-entropy and the Natural R\'{e}nyi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial…
We present the Shannon entropy as an indicator of spatial resolution for morphology of resonance mode pattern in dielectric micro cavity. We obtain two types of optimized mesh point for the minimum and maximum sizes, respectively. The…
In this paper, we investigate the asymptotic stability of finite-dimensional stochastic integrable Hamiltonian systems via information entropy. Specifically, we establish the asymptotic vanishing of Shannon entropy difference (with…
We propose a new way of defining entropy of a system, which gives a general form which may be nonextensive as Tsallis entropy, but is linearly dependent on component entropies, like Renyi entropy, which is extensive. This entropy has a…
Many axiomatic definitions of entropy, such as the R\'enyi entropy, of a random variable are closely related to the $\ell_{\alpha}$-norm of its probability distribution. This study considers probability distributions on finite sets, and…
Recently, in [P. Jizba and J. Korbel, Physica A 444, 2016, 808-827], four generalized Shannon-Khinchin [GSK] axioms have been proposed and a generalized entropy which uniquely satisfies the GSK axioms has been derived. In this comment, we…
By using the Renyi entropy, and following the same scheme that in the Fisher-Renyi entropy product case, a generalized statistical complexity is defined. Several properties of it, including inequalities and lower and upper bounds are…
Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove…
Recent years have seen the rise of convolutional neural network techniques in exemplar-based image synthesis. These methods often rely on the minimization of some variational formulation on the image space for which the minimizers are…
In this paper, we prove that the Renyi entropy of linearly normalized partial maxima of independent and identically distributed random variables is convergent to the corresponding limit Renyi entropy when the linearly normalized partial…
Although maximum entropy method (maxEnt method) is currently the standard algorithm for extracting real frequency information from imaginary frequency Green function, still this method is beset with overfitting problem, which manifests…
We describe a quantum algorithm to estimate the $\alpha$-Renyi entropy of an unknown density matrix $\rho\in\mathcal{C}^{d\times d}$ for $\alpha\neq 1$ by combining the recent technique of quantum singular value transformations with the…
We apply the Principle of Maximum Entropy to the study of a general class of deterministic fractal sets. The scaling laws peculiar to these objects are accounted for by means of a constraint concerning the average content of information in…
This paper studies sample average approximation (SAA) in solving convex or strongly convex stochastic programming (SP) problems. In estimating SAA's sample efficiency, the state-of-the-art sample complexity bounds entail metric entropy…
An experiment to study the entropy method for an anomaly detection system has been performed. The study has been conducted using real data generated from the distributed sensor networks at the Intel Berkeley Research Laboratory. The…
Using the concept of discrete noiseless channels, it was shown by Shannon in A Mathematical Theory of Communication that the ultimate performance of an encoder for a constrained system is limited by the combinatorial capacity of the system…
Making statistical predictions requires tackling two problems: one must assign appropriate probability distributions and then one must calculate a variety of expected values. The method of maximum entropy is commonly used to address the…
We present results on the concentration properties of the spectral norm $\|A_p\|$ of the adjacency matrix $A_p$ of an Erd\H{o}s-R\'enyi random graph $G(n,p)$. First we consider the Erd\H{o}s-R\'enyi random graph process and prove that…