Related papers: Point Divergence Gain and Multidimensional Data Se…
The R\'enyi entropy is a generalization of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic R\'enyi…
We construct the entropic measure $\mathbb{P}^\beta$ on compact manifolds of any dimension. It is defined as the push forward of the Dirichlet process (another random probability measure, well-known to exist on spaces of any dimension)…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
Semi-continuous data comes from a distribution that is a mixture of the point mass at zero and a continuous distribution with support on the positive real line. A clear example is the daily rainfall data. In this paper, we present a novel…
As a part of the construction of an information theory based on general probabilistic theories, we propose and investigate the several distinguishability measures and "entropies" in general probabilistic theories. As their applications,…
Two identities in statistical mechanics involving entropy differences (or ratios of density of states) at constant energy are derived. The first provides a nontrivial extension of the Jarzynski equality to the microcanonical ensemble [C.…
We study the behavior of the random walk in a continuum independent long-range percolation model, in which two given vertices $x$ and $y$ are connected with probability that asymptotically behaves like $|x-y|^{-\alpha}$ with $\alpha>d$,…
We introduce a novel method, called Dispersion Entropy for Graph Signals, $DE_G$, as a powerful tool for analysing the irregularity of signals defined on graphs. We demonstrate the effectiveness of $DE_G$ in detecting changes in the…
Probability distributions and densities are derived for the excess and deficiency of the intensity or instantaneous energy (quasi-static power) associated with a $p$-dimensional random vector field. Explicit expressions for the exact…
Given a heterogeneous time-series sample, the objective is to find points in time (called change points) where the probability distribution generating the data has changed. The data are assumed to have been generated by arbitrary unknown…
Entropy arises in strong interactions by a dynamical separation of ``partons'' from unobservable ``environment'' modes due to confinement. For interacting scalar fields we calculate the statistical entropy of the observable subsystem.…
Mixture distributions are extensively used as a modeling tool in diverse areas from machine learning to communications engineering to physics, and obtaining bounds on the entropy of probability distributions is of fundamental importance in…
We extend a theorem of Maa, Pearl, and Bartoszynski, which links equality of interpoint distance distributions to equality of underlying multivariate distributions, beyond the restrictive class of homogeneous, translation-invariant distance…
Nonlinear dynamics play an important role in the analysis of signals. A popular, readily interpretable nonlinear measure is Permutation Entropy. It has recently been extended for the analysis of graph signals, thus providing a framework for…
Information theoretic quantities play a central role in machine learning. The recent surge in the complexity of data and models has increased the demand for accurate estimation of these quantities. However, as the dimension grows the…
We develop an information-theoretic approach to isoperimetric inequalities based on entropy dissipation under heat flow. By viewing diffusion as a noisy information channel, we measure how mutual information about set membership decays over…
We extend previously proposed measures of complexity, emergence, and self-organization to continuous distributions using differential entropy. This allows us to calculate the complexity of phenomena for which distributions are known. We…
An extension of the entropy power inequality to the form $N_r^\alpha(X+Y) \geq N_r^\alpha(X) + N_r^\alpha(Y)$ with arbitrary independent summands $X$ and $Y$ in $\mathbb{R}^n$ is obtained for the R\'enyi entropy and powers $\alpha \geq…
We introduce Multivariate Multiscale Graph-based Dispersion Entropy (mvDEG), a novel, computationally efficient method for analyzing multivariate time series data in graph and complex network frameworks, and demonstrate its application in…
This letter reports two moment extensions of the entropy of a distribution. By understanding the traditional entropy as the average of the original distribution up to a random variable transformation, the traditional moments equation become…