Related papers: Entropy and the Discrete Central Limit Theorem

Define the non-overlapping return time of a random process to be the number of blocks that we wait before a particular block reappears. We prove a Central Limit Theorem based on these return times. This result has applications to entropy…

Relative entropy is a powerful measure of the dissimilarity between two statistical field theories in the continuum. In this work, we study the relative entropy between Gaussian scalar field theories in a finite volume with different masses…

Let $\{V_{i,j}; (i,j)\in\N^2\}$ be a two-dimensional array of i.i.d.\ random variables. The limit laws of the sum of independent random products $$ Z_n=\sum_{i=1}^{N_n} \prod_{j=1}^{n} e^{V_{i,j}} $$ as $n,N_n\to\infty$ have been…

We introduce a framework to study discrete-variable (DV) quantum systems based on qudits. It relies on notions of a mean state (MS), a minimal stabilizer-projection state (MSPS), and a new convolution. Some interesting consequences are: The…

Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…

We introduce a new generalization of relative entropy to non-negative vectors with sums $\gt 1$. We show in a purely combinatorial setting, with no probabilistic considerations, that in the presence of linear constraints defining a convex…

We investigate the dependence of the center-of-mass tomogram of a system with many degrees of freedom $N$ on the Planck constant $\hbar $. It is shown that to use the central limit theorem under taking the limit $N\to +\infty $ one should…

Sharp, nonasymptotic bounds are obtained for the relative entropy between the distributions of sampling with and without replacement from an urn with balls of $c\geq 2$ colors. Our bounds are asymptotically tight in certain regimes and,…

Information theory on a time-discrete setting in the framework of time series analysis is generalized to the time-continuous case. Considerations of the Roessler and Lorenz dynamics as well as the Ornstein-Uhlenbeck process yield for…

In nature or societies, the power-law is present ubiquitously, and then it is important to investigate the mathematical characteristics of power-laws in the recent era of big data. In this paper we prove the superposition of non-identical…

A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is…

Let $\mathsf{N}_{\rm d}\left[X\right]=\frac{1}{2\pi {\rm e}}{\rm e}^{2\mathsf{H}\left[X\right]}$ denote the entropy power of the discrete random variable $X$ where $\mathsf{H}\left[X\right]$ denotes the discrete entropy of $X$. In this…

The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…

We consider the existence of the integrated density of states (IDS) of the Anderson model on the Hilbert space $\ell^2(\mathbb{Z}^d)$ as analogues to the law of large numbers (LLN). In this work, we prove the analogues central limit theorem…

By the Lindeberg-L\'evy central limit theorem, standardized partial sums of a sequence of mutually independent and identically distributed random variables converge in law to the standard normal distribution. It is known that mutual…

Entropy, its production, and its change in a dynamical system can be understood from either a fully stochastic dynamic description or from a deterministic dynamics exhibiting chaotic behavior. By taking the former approach based on the…

This paper derives central limit and bootstrap theorems for probabilities that sums of centered high-dimensional random vectors hit hyperrectangles and sparsely convex sets. Specifically, we derive Gaussian and bootstrap approximations for…

We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…

A system of $N$ weakly interacting particles whose dynamics is given in terms of jump-diffusions with a common factor is considered. The common factor is described through another jump-diffusion and the coefficients of the evolution…

The time variation of entropy, as an alternative to the variance, is proposed as a measure of the diffusion rate. It is shown that for linear and time-translationally invariant systems having a large-time limit for the density, at large…