Related papers: Information inequalities and a dependent Central L…
We adapt arguments concerning entropy-theoretic convergence from the independent case to the case of FKG random variables. FKG systems are chosen since their dependence structure is controlled through covariance alone, though in the sequel…
"Bounds on information combining" are entropic inequalities that determine how the information (entropy) of a set of random variables can change when these are combined in certain prescribed ways. Such bounds play an important role in…
We deduce sufficient conditions for the Central Limit Theorem (CLT) in the Lebesgue-Riesz space L(p) defined on some measure space for the sequence of centered random variables satisfying the strong mixing (Rosenblatt) condition. We…
The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…
We consider the central limit theorem for stable laws in the case of the standardized sum of independent and identically distributed random variables with regular probability density function. By showing decay of different entropy…
New families of Fisher information and entropy power inequalities for sums of independent random variables are presented. These inequalities relate the information in the sum of $n$ independent random variables to the information contained…
Bounds on information combining are entropic inequalities that determine how the information, or entropy, of a set of random variables can change when they are combined in certain prescribed ways. Such bounds play an important role in…
An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic…
A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of…
We give conditions for an O(1/n) rate of convergence of Fisher information and relative entropy in the Central Limit Theorem. We use the theory of projections in L2 spaces and Poincare inequalities, to provide a better understanding of the…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
In this paper, we derive a central limit theorem for collections of weakly correlated random variables indexed by discrete metric spaces, where the correlation decays in the distance of the indices. The correlation structure we study…
The main result of the article is the rate of convergence to the Rosenblatt-type distributions in non-central limit theorems. Specifications of the main theorem are discussed for several scenarios. In particular, special attention is paid…
The Central Limit Theorem states that, in the limit of a large number of terms, an appropriately scaled sum of independent random variables yields another random variable whose probability distribution tends to a stable distribution. The…
This paper extends classical probabilistic results to the broader class of demimartingales and demisubmartingales. We establish variants of Doob's-type optional sampling theorem under minimal structural conditions on stopping times, relying…
We consider the behaviour of the Fisher information of scaled sums of independent and identically distributed random variables in the Central Limit Theorem regime. We show how this behaviour can be related to the second-largest non-trivial…
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
A central limit theorem is established for a sum of random variables belonging to a sequence of random fields. The fields are assumed to have zero mean conditional on the past history and to satisfy certain conditional $\alpha$-mixing…
Constraints on entropies are considered to be the laws of information theory. Even though the pursuit of their discovery has been a central theme of research in information theory, the algorithmic aspects of constraints on entropies remain…
The theorem of Shannon-McMillan-Breiman states that for every generating partition on an ergodic system, the exponential decay rate of the measure of cylinder sets equals the metric entropy almost everywhere (provided the entropy is…