Related papers: Statistical estimation of the Shannon entropy
In this paper, we present a theoretical discussion on AI deep learning neural network uncertainty investigation based on the classical Rademacher complexity and Shannon entropy. First it is shown that the classical Rademacher complexity and…
We study the convergence analysis for general degenerate and non-reversible stochastic differential equations (SDEs). We apply the Lyapunov method to analyze the Fokker-Planck equation, in which the Lyapunov functional is chosen as a…
This work deals with the scattering entropy of quantum graphs in many different circumstances. We first consider the case of the Shannon entropy and then the R\'enyi and Tsallis entropies, which are more adequate to study distinct…
An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic…
Shannon's entropy and other entropy-based concepts are derived from the new, more general concept of relative divergence of one "grading' function on a linearly ordered set from another such function. The definition of relative divergence…
We present some new nonparametric estimators of entropies and we establish almost sure consistency and central limit Theorems for some of the most important entropies in the discrete case. Our theorical results are validated by simulations.
The uncertainty relation based on the Shannon entropies of the probability densities in position and momentum spaces is improved for quantum systems in arbitrary $D$-dimensional spherically symmetric potentials. To find it, we have used the…
Shannon entropy in position ($S_{\rvec}$) and momentum ($S_{\pvec}$) spaces, along with their sum ($S_t$) are presented for unit-normalized densities of He, Li$^+$ and Be$^{2+}$ ions, spatially confined at the center of an impenetrable…
We provide experimental validation of tight entropic uncertainty relations for the Shannon entropies of observables with mutually unbiased eigenstates in high dimensions. In particular, we address the cases of dimensions $d = 3$, $4$ and…
We study the linear Zakharov--Kuznetsov equation with periodic boundary conditions. Employing some tools from the nonharmonic Fourier series we obtain several internal observability theorems. Then we prove various exact controllability and…
We show that for weakly dependent random variables the relative entropy functional satisfies an approximate version of the standard tensorization property which holds in the independent case. As a corollary we obtain a family of…
Within a framework of utmost generality, we show that the entropy maximization procedure with linear constraints uniquely leads to the Shannon-Boltzmann-Gibbs entropy. Therefore, the use of this procedure with linear constraints should not…
Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…
The Shannon entropy is a widely used summary statistic, for example, network traffic measurement, anomaly detection, neural computations, spike trains, etc. This study focuses on estimating Shannon entropy of data streams. It is known that…
Entropy functionals are computed for non-stationary distributions of particles of Lorentz gas and hard disks. The distributions consisting of beams of particles are found to have the largest amount of entropy and entropy increase. The…
We exhibit infinitely many new, constrained inequalities for the von Neumann entropy, and show that they are independent of each other and the known inequalities obeyed by the von Neumann entropy (basically strong subadditivity). The new…
We present a numerical study of the application of the Shannon entropy technique to the planar restricted three-body problem in the vicinity of first-order interior mean-motion resonances with the perturber. We estimate the diffusion…
Entropy and its various generalizations are important in many fields, including mathematical statistics, communication theory, physics and computer science, for characterizing the amount of information associated with a probability…
We prove the logarithmic convexity of certain quantities, which measure the quadratic exponential decay at infinity and within two characteristic hyperplanes of solutions of Schr\"odinger evolutions. As a consequence we obtain some…
We prove that if two free p.m.p. $\mathbb{Z}$-actions are Shannon orbit equivalent then they have the same entropy. The argument also applies more generally to yield the same conclusion for free p.m.p. actions of finitely generated…