Related papers: Dimension-Free Empirical Entropy Estimation

In this paper, we identify a class of absolutely continuous probability distributions, and show that the differential entropy is uniformly convergent over this space under the metric of total variation distance. One of the advantages of…

Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…

We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…

Language models (LMs) are trained on billions of tokens in an attempt to recover the true language distribution. Still, vanilla random sampling from LMs yields low quality generations. Decoding algorithms attempt to restrict the LM…

We consider word complexity and topological entropy for random substitution subshifts. In contrast to previous work, we do not assume that the underlying random substitution is compatible. We show that the subshift of a primitive random…

The question of optimally approximating an arbitrary probability measure in the Wasserstein distance by a discrete one with uniform weights is considered. Estimates are obtained for the optimal approximation distance, with an explicit rate…

We examine entropy production in reduced descriptions of collisionless plasmas. Introducing a closure dependent entropy hierarchy, we show that non-conservative moment closures generate a residual entropy associated with irreversible…

We discuss information-theoretic concepts on infinite-dimensional quantum systems. In particular, we lift the smooth entropy formalism as introduced by Renner and collaborators for finite-dimensional systems to von Neumann algebras. For the…

Obtaining guarantees on the convergence of the minimizers of empirical risks to the ones of the true risk is a fundamental matter in statistical learning. Instead of deriving guarantees on the usual estimation error, the goal of this paper…

In this paper we show that the existence of a primarily discrete space-time may be a fruitful assumption from which we may develop a new approach of statistical thermodynamics in pre-relativistic conditions. The discreetness of space-time…

We consider some classes of piecewise expanding maps in finite dimensional spaces having invariant probability measures which are absolutely continuous with respect to Lebesgue measure. We derive an entropy formula for such measures and,…

We study the existing algorithms that solve the multidimensional martingale optimal transport. Then we provide a new algorithm based on entropic regularization and Newton's method. Then we provide theoretical convergence rate results and we…

We introduce the problem of \emph{entropy equivalence testing} for probability distributions, a relaxation of the well-studied closeness testing problem, where the distribution testing algorithm is now only required to distinguish, given…

This note attempts to revisit the classical results on Laplace approximation in a modern non-asymptotic and dimension free form. Such an extension is motivated by applications to high dimensional statistical and optimization problems. The…

In a recent paper, the authors proposed a general methodology for probabilistic learning on manifolds. The method was used to generate numerical samples that are statistically consistent with an existing dataset construed as a realization…

Estimation of Shannon and R\'enyi entropies of unknown discrete distributions is a fundamental problem in statistical property testing and an active research topic in both theoretical computer science and information theory. Tight bounds on…

We prove an exponential deviation inequality for the convex hull of a finite sample of i.i.d. random points with a density supported on an arbitrary convex body in $\R^d$, $d\geq 2$. When the density is uniform, our result yields rate…

The problem of assigning probability distributions which objectively reflect the prior information available about experiments is one of the major stumbling blocks in the use of Bayesian methods of data analysis. In this paper the method of…

The study considers advantages of the introduced measure of time based on the entropy change under irreversible processes (entropy production). Using the example of non-equilibrium expansion of an ideal gas in vacuum, such a measure is…

We derive an equilibrated a posteriori error estimator for the space (semi) discretization of the scalar wave equation by finite elements. In the idealized setting where time discretization is ignored and the simulation time is large, we…