Related papers: Metric entropy for functions of bounded total gene…
Transfer entropy is a widely used measure for quantifying directed information flows in complex systems. While the challenges of estimating transfer entropy for continuous data are well known, it has two major shortcomings for data of…
We consider the one-parameter family of interval maps arising from generalized continued fraction expansions known as alpha-continued fractions. For such maps, we perform a numerical study of the behaviour of metric entropy as a function of…
We describe in great generality features concerning constrained entropic, functional variational problems that allow for a broad range of applications. Our discussion encompasses not only entropies but, potentially, any functional of the…
Mixture distributions are a workhorse model for multimodal data in information theory, signal processing, and machine learning. Yet even when each component density is simple, the differential entropy of the mixture is notoriously hard to…
We develop efficient binary (i.e., 1-bit) and multi-bit coding schemes for estimating the scale parameter of $\alpha$-stable distributions. The work is motivated by the recent work on one scan 1-bit compressed sensing (sparse signal…
Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…
This paper presents the notion of a variation entropy. This concept is an entropy framework for the gradient of the solution of a conservation law instead of on the solution itself. It appears that all semi-norms are admissible variation…
We consider distributions of ordered random vectors with given one-dimensional marginal distributions. We give an elementary necessary and sufficient condition for the existence of such a distribution with finite entropy. In this case, we…
One way of getting insight into non-Gaussian measures, posed on infinite dimensional Hilbert spaces, is to first obtain best fit Gaussian approximations, which are more amenable to numerical approximation. These Gaussians can then be used…
Two estimates for the inverse binary entropy function are derived using the property of information entropy to estimate combinatorics of sequences as well as related formulas from population genetics for the effective number of alleles. The…
Information entropy and its extension, which are important generalization of entropy, have been applied in many research domains today. In this paper, a novel generalized relative entropy is constructed to avoid some defects of traditional…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…
The geometric Brownian motion (GBM) is widely employed for modeling stochastic processes, yet its solutions are characterized by the log-normal distribution. This comprises predictive capabilities of GBM mainly in terms of forecasting…
It is well-known that the relativized variational principle established by Bogenschutz and Kifer connects the fiber topological entropy and fiber measure-theoretic entropy. In context of random dynamical systems, metric mean dimension was…
We prove general results about separation and weak$^\#$-convergence of boundedly finite measures on separable metric spaces and Souslin spaces. More precisely, we consider an algebra of bounded real-valued, or more generally a $*$-algebra…
We study the approximation of expectations $\E(f(X))$ for Gaussian random elements $X$ with values in a separable Hilbert space $H$ and Lipschitz continuous functionals $f \colon H \to \R$. We consider restricted Monte Carlo algorithms,…
We construct the generalized entropy optimized by a given arbitrary statistical distribution with a finite linear expectation value of a random quantity of interest. This offers, via the maximum entropy principle, a unified basis for a…
Binary embedding is a nonlinear dimension reduction methodology where high dimensional data are embedded into the Hamming cube while preserving the structure of the original space. Specifically, for an arbitrary $N$ distinct points in…
We provide a quantum statistical basis for (a)a characterisation of a complete set of thermodynamic variables and (b) the differentiability of the entropy function of these variables
We show that a simple geometric result suffices to derive the form of the optimal solution in a large class of finite and infinite-dimensional maximum entropy problems concerning probability distributions, spectral densities and covariance…