Related papers: Explicit Bounds for Entropy Concentration under Li…
We revisit the maximum-entropy inference of the state of a finite-level quantum system under linear constraints. The constraints are specified by the expected values of a set of fixed observables. We point out the existence of…
Traditionally, the Method of (Shannon-Kullback's) Relative Entropy Maximization (REM) is considered with linear moment constraints. In this work, the method is studied under frequency moment constraints which are non-linear in…
We obtain non asymptotic bounds for the Monte Carlo algorithm associated to the Euler discretization of some diffusion processes. The key tool is the Gaussian concentration satisfied by the density of the discretization scheme. This…
Bayes' theorem incorporates distinct types of information through the likelihood and prior. Direct observations of state variables enter the likelihood and modify posterior probabilities through consistent updating. Information in terms of…
We study the relative entropy between the empirical estimate of a discrete distribution and the true underlying distribution. If the minimum value of the probability mass function exceeds an $\alpha > 0$ (i.e. when the true underlying…
In the present paper, we derive Berry-Esseen bounds for the estimation of diversity indices on countable alphabets. A general non-asymptotic convergence rate is established for the plug-in estimator of a wide class of indices, including…
Several implications of well-known fluctuation theorems, on the statistical properties of the entropy production, are studied using various approaches. We begin by deriving a tight lower bound on the variance of the entropy production for a…
Entropy quantifies the number of bits required to store objects under certain given assumptions. While this is a well established concept for strings, in the context of tries the state-of-the-art regarding entropies is less developed. The…
The method of maximum entropy (ME) is extended to address the following problem: Once one accepts that the ME distribution is to be preferred over all others, the question is to what extent are distributions with lower entropy supposed to…
In this work the $\ell_q$-norms of points chosen uniformly at random in a centered regular simplex in high dimensions are studied. Berry-Esseen bounds in the regime $1\leq q < \infty$ are derived and complemented by a non-central limit…
This note is concerned with weakly interacting stochastic particle systems with possibly singular pairwise interactions. In this setting, we observe a connection between entropic propagation of chaos and exponential concentration bounds for…
The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…
Given a sequence composed of a limit number of characters, we try to "read" it as a "text". This involves to segment the sequence into "words". The difficulty is to distinguish good segmentation from enormous number of random ones.Aiming at…
A novel principle is presented which allows for the proof of bounded weak solutions to a class of physically relevant, strongly coupled parabolic systems exhibiting a formal gradient-flow structure. The main feature of these systems is that…
We establish fundamental limits on estimation accuracy for the noisy 20 questions problem with measurement-dependent noise and introduce optimal non-adaptive procedures that achieve these limits. The minimal achievable resolution is defined…
It is shown that a consistent application of Bayesian updating from a prior probability density to a posterior using evidence in the form of expectation constraints leads to exactly the same results as the application of the maximum entropy…
Let $X_1,\ldots,X_N$ be i.i.d.\ random variables distributed like $X$. Suppose that the first $k \geq 3$ moments $\{ \mathbb{E}[X^j] : j = 1,\ldots,k\}$ of $X$ agree with that of the standard Gaussian distribution, that…
We prove a Berry-Esseen theorem, a local central limit theorem and (local) large and (global) moderate deviations principles for i.i.d. (uniformly) random non-uniformly expanding or hyperbolic maps with exponential first return times. Using…
The fluctuation theorem for entropy production is a remarkable symmetry of the distribution of produced entropy that holds universally in non-equilibrium steady states with Markovian dynamics. However, in systems with slow degrees of…
This paper provides an extensive study of the behavior of the best achievable rate (and other related fundamental limits) in variable-length lossless compression. In the non-asymptotic regime, the fundamental limits of fixed-to-variable…