Related papers: Metric entropy for functions of bounded total gene…
We obtain bounds on fluctuations of two entropy estimators for a class of one-dimensional Gibbs measures on the full shift. They are the consequence of a general exponential inequality for Lipschitz functions of n variables. The first…
Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a…
The notion of metric entropy dimension is introduced to measure the complexity of entropy zero dynamical systems. For measure preserving systems, we define entropy dimension via the dimension of entropy generating sequences. This…
The method of maximum entropy (ME) is extended to address the following problem: Once one accepts that the ME distribution is to be preferred over all others, the question is to what extent are distributions with lower entropy supposed to…
In this paper, we study functions of bounded variation on a complete and connected metric space with finite one-dimensional Hausdorff measure. The definition of BV functions on a compact interval based on pointwise variation is extended to…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
Solutions to conservation laws satisfy the monotonicity property: the number of local extrema is a non-increasing function of time, and local maximum/minimum values decrease/increase monotonically in time. This paper investigates this…
Measure-theoretic slow entropy is a more refined invariant than the classical measure-theoretic entropy to characterize the complexity of dynamical systems with subexponential growth rates of distinguishable orbit types. In this paper we…
In applications with significant class imbalance or asymmetric costs, metrics such as the $F_\beta$-measure, AM measure, Jaccard similarity coefficient, and weighted accuracy offer more suitable evaluation criteria than standard binary…
In its continuous version, the entropy functional measuring the information content of a given probability density may be plagued by a "measure" problem that results from improper weighting of phase space. This issue is addressed…
Mixture distributions are extensively used as a modeling tool in diverse areas from machine learning to communications engineering to physics, and obtaining bounds on the entropy of probability distributions is of fundamental importance in…
We consider the problem of estimating the population probability distribution given a finite set of multivariate samples, using the maximum entropy approach. In strict keeping with Jaynes' original definition, our precise formulation of the…
We revisit the classical problem of inverting dimension-reducing linear mappings using the maximum entropy (MaxEnt) criterion. In the literature, solutions are problem-dependent, inconsistent, and use different entropy measures. We propose…
This work contains two single-letter upper bounds on the entropy rate of a discrete-valued stationary stochastic process, which only depend on second-order statistics, and are primarily suitable for models which consist of relatively large…
We describe a maximum entropy approach for computing volumes and counting integer points in polyhedra. To estimate the number of points from a particular set X in R^n in a polyhedron P in R^n, by solving a certain entropy maximization…
Coarse geometry studies metric spaces on the large scale. The recently introduced notion of coarse entropy is a tool to study dynamics from the coarse point of view. We prove that all isometries of a given metric space have the same coarse…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
We study the relative entropy density for generalized Gibbs measures. We first show its existence and obtain a familiar expression in terms of entropy and relative energy for a class of ``almost Gibbsian measures'' (almost sure continuity…
The phenomenon of entropy concentration provides strong support for the maximum entropy method, MaxEnt, for inferring a probability vector from information in the form of constraints. Here we extend this phenomenon, in a discrete setting,…
In this document, we aim to gather various results related to a compositional/categorical approach to rigorous Statistical Mechanics. Rigorous Statistical Mechanics is centered on the mathematical study of statistical systems. Central…