Related papers: On the equivalence between Stein and de Bruijn ide…
In this paper, we propose generalizations of the de Bruijn's identities based on extensions of the Shannon entropy, Fisher information and their associated divergences or relative measures. The foundation of these generalizations are the…
Motivated by the classical De Bruijn's identity for the additive Gaussian noise channel, in this paper we consider a generalized setting where the channel is modelled via stochastic differential equations driven by fractional Brownian…
We show that an information-theoretic relation called the de Bruijn-type identity can be reformulated in a physical context with probability currents. The time derivatives of relative entropies under the continuity equation are presented,…
We show how some attractive information--theoretic properties of Gaussians pass over to more general families of stable densities. We define a new score function for symmetric stable laws, and use it to give a stable version of the heat…
In this paper, we introduce new Stein identities for gamma target distribution as well as a new non-linear channel specifically designed for gamma inputs. From these two ingredients, we derive an explicit and simple formula for the…
We provide a new perspective on Stein's so-called density approach by introducing a new operator and characterizing class which are valid for a much wider family of probability distributions on the real line. We prove an elementary…
Stein's identity is a fundamental tool in machine learning with applications in generative models, stochastic optimization, and other problems involving gradients of expectations under Gaussian distributions. Less attention has been paid to…
Building on the recently introduced notion of Boolean entropy, we define the corresponding Boolean Fisher information via a de Bruijn identity. We study the monotonicity of this Fisher information in the Boolean Central Limit Theorem and…
In this communication, we describe some interrelations between generalized $q$-entropies and a generalized version of Fisher information. In information theory, the de Bruijn identity links the Fisher information and the derivative of the…
Stein's method is applied to obtain a general Cramer-type moderate deviation result for dependent random variables whose dependence is defined in terms of a Stein identity. A corollary for zero-bias coupling is deduced. The result is also…
This paper is a short exposition of Stein's method of normal approximation from my personal perspective. It focuses mainly on the characterization of the normal distribution and the construction of Stein identities. Through examples, it…
We investigate free-energy dissipation in a continuous-time birth-and-death dynamics in $\mathbb{R}^d$. For these Markov processes, the class of reversible measures coincides with the infinite-volume Gibbs point processes for some…
We generalize the so-called density approach to Stein characterizations of probability distributions. We prove an elementary factorization property of the resulting Stein operator in terms of a generalized (standardized) score function. We…
While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, up to now Shannon's entropy power inequality (EPI) is an exception: Existing information theoretic proofs of the…
The de Bruijn identity states that Fisher information is equal to a half of the time-derivative of Shannon differential entropy along heat flow. In the same spirit, a generalized version of Fisher information, which we term the…
Consider a channel ${\bf Y}={\bf X}+ {\bf N}$ where ${\bf X}$ is an $n$-dimensional random vector, and ${\bf N}$ is a Gaussian vector with a covariance matrix ${\bf \mathsf{K}}_{\bf N}$. The object under consideration in this paper is the…
We consider estimating the parametric components of semi-parametric multiple index models in a high-dimensional and non-Gaussian setting. Such models form a rich class of non-linear models with applications to signal processing, machine…
We show that there is equality in Shannon's Entropy Power Inequality (EPI) if and only if the random variables involved are Gaussian, assuming nothing beyond the existence of differential entropies. This is done by justifying de Bruijn's…
Pinsker's inequality states that the relative entropy $d_{\mathrm{KL}}(X, Y)$ between two random variables $X$ and $Y$ dominates the square of the total variation distance $d_{\mathrm{TV}}(X,Y)$ between $X$ and $Y$. In this paper we…
The derivation and application of Stein identities have received considerable research interest in recent years, especially for continuous or discrete-univariate distributions. In this paper, we complement the existing literature by…