相关论文: On Divergence-Power Inequalities

We offer a new point of view on the (Modified) Log-Sobolev inequality and lower bounds on the Ricci-curvature in the setting where the dynamics are obtained as the limit of Markov processes. In this setting, the large deviation rate…

A quantity of interest to characterise continuous-valued stochastic processes is the differential entropy rate. The rate of convergence of many properties of LRD processes is slower than might be expected, based on the intuition for…

We present the Shannon entropy as an indicator of spatial resolution for morphology of resonance mode pattern in dielectric micro cavity. We obtain two types of optimized mesh point for the minimum and maximum sizes, respectively. The…

In this paper, I expand Shannon's definition of entropy into a new form of entropy that allows integration of information from different random events. Shannon's notion of entropy is a special case of my more general definition of entropy.…

Shannon information entropy is a natural measure of probability (de)localization and thus (un)predictability in various procedures of data analysis for model systems. We pay particular attention to links between the Shannon entropy and the…

The estimation of entropy rates for stationary discrete-valued stochastic processes is a well studied problem in information theory. However, estimating the entropy rate for stationary continuous-valued stochastic processes has not received…

We consider transmission of stationary and ergodic sources over non-ergodic composite channels with channel state information at the receiver (CSIR). Previously we introduced alternate capacity definitions to Shannon capacity, including the…

This work explores the theoretical and practical foundations of denoising diffusion probabilistic models (DDPMs) and score-based generative models, which leverage stochastic processes and Brownian motion to model complex data distributions.…

Within the framework of linear vector Gaussian channels with arbitrary signaling, closed-form expressions for the Jacobian of the minimum mean square error and Fisher information matrices with respect to arbitrary parameters of the system…

For a family of stochastic differential equations, we investigate the asymptotic behaviors of its corresponding Picard's iteration, establishing convergence results in terms of relative entropy. Our convergence results complement the…

We provide a condition under which a version of Shannon's Entropy Power Inequality will hold for dependent variables. We provide information inequalities extending those found in the independent case.

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential…

A Massey-like inequality is any useful lower bound on guessing entropy in terms of the computationally scalable Shannon entropy. The asymptotically optimal Massey-like inequality is determined and further refined for finite-support…

New families of Fisher information and entropy power inequalities for sums of independent random variables are presented. These inequalities relate the information in the sum of $n$ independent random variables to the information contained…

A statistical model of discrete finite length random processes with negative power law spectral densities is presented. The definition of terms is followed by a description of the spectral density trend. An algorithmic construction of…

We study the Shannon entropy of the probability distribution resulting from the ground-state wave function of a one-dimensional quantum model. This entropy is related to the entanglement entropy of a Rokhsar-Kivelson-type wave function…

Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…

Power iteration has been generalized to solve many interesting problems in machine learning and statistics. Despite its striking success, theoretical understanding of when and how such an algorithm enjoys good convergence property is…

We consider the problem of distributed estimation, where local processors observe independent samples conditioned on a common random parameter of interest, map the observations to a finite number of bits, and send these bits to a remote…

By developing a new technique called the bi-coupling argument, we estimate the relative entropy between different diffusion processes in terms of the distances of initial distributions and drift-diffusion coefficients. As an application,…