Related papers: Metric entropy for functions of bounded total gene…
It is shown that for any ensemble, whether classical or quantum, continuous or discrete, there is only one measure of the "volume" of the ensemble that is compatible with several basic geometric postulates. This volume measure is thus a…
We prove a lower bound on the relative entropy between two finite-dimensional states in terms of their entropy difference and the dimension of the underlying space. The inequality is tight in the sense that equality can be attained for any…
In finite mixture models, apart from underlying mixing measure, true kernel density function of each subpopulation in the data is, in many scenarios, unknown. Perhaps the most popular approach is to choose some kernel functions that we…
Error entropy is a important nonlinear similarity measure, and it has received increasing attention in many practical applications. The default kernel function of error entropy criterion is Gaussian kernel function, however, which is not…
Semi-continuous data comes from a distribution that is a mixture of the point mass at zero and a continuous distribution with support on the positive real line. A clear example is the daily rainfall data. In this paper, we present a novel…
In this paper we consider continued fraction (CF) expansions on intervals different from $[0,1]$. For every $x$ in such interval we find a CF expansion with a finite number of possible digits. Using the natural extension, the density of the…
Motivated by the approach of random linear codes, a new distance in the vector space over a finite field is defined as the logarithm of the "surface area" of a Hamming ball with radius being the corresponding Hamming distance. It is named…
In Bayesian inference, we seek to compute information about random variables such as moments or quantiles on the basis of {available data} and prior information. When the distribution of random variables is {intractable}, Monte Carlo (MC)…
The definition of weighted entropy allows for easy calculation of the entropy of the mixture of measures. In this paper we investigate the problem of equivalent definition of the general entropy function in weighted form. We show that under…
Using entropic inequalities from information theory, we provide new bounds on the total variation and 2-Wasserstein distances between a conditionally Gaussian law and a Gaussian law with invertible covariance matrix. We apply our results to…
We study a quantity called discrete layered entropy, which approximates the Shannon entropy within a logarithmic gap. Compared to the Shannon entropy, the discrete layered entropy is piecewise linear, approximates the expected length of the…
We consider systems of conservation laws endowed with a convex entropy. We show the contraction, up to a translation, to extremal entropic shocks, for a pseudo-distance based on the notion of relative entropy. The contraction holds for…
We present a series of closed-form maximum entropy upper bounds for the differential entropy of a continuous univariate random variable and study the properties of that series. We then show how to use those generic bounds for upper bounding…
Entropy is one of the most fundamental quantities in physics. For systems with few degrees of freedom, the value of entropy provides a powerful insight into its microscopic dynamics, such as the number, degeneracy and relative energies of…
Inspired by logistic regression, we introduce a regression model for data tuples consisting of a binary response and a set of covariates residing in a metric space without vector structures. Based on the proposed model we also develop a…
It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight…
This paper introduces a comprehensive framework for complex-valued probability measures and explores their novel applications in information theory and statistical analysis. We define a complex probability measure as a phase-modulated…
We study the problem of representing all distances between $n$ points in $\mathbb R^d$, with arbitrarily small distortion, using as few bits as possible. We give asymptotically tight bounds for this problem, for Euclidean metrics, for…
We use discontinuous molecular dynamics and grand-canonical transition-matrix Monte Carlo simulations to explore how confinement between parallel hard walls modifies the relationships between packing fraction, self-diffusivity, partial…
Permutation entropy measures the complexity of deterministic time series via a data symbolic quantization consisting of rank vectors called ordinal patterns or just permutations. The reasons for the increasing popularity of this entropy in…