Related papers: Information inequalities and a dependent Central L…
Mixture distributions are a workhorse model for multimodal data in information theory, signal processing, and machine learning. Yet even when each component density is simple, the differential entropy of the mixture is notoriously hard to…
We investigate the concept of entropy in probabilistic theories more general than quantum mechanics, with particular reference to the notion of information causality recently proposed by Pawlowski et. al. (arXiv:0905.2992). We consider two…
We give a simple and general central limit theorem for a triangular array of m-dependent variables. The result requires only a Lindeberg condition and avoids unnecessary extra conditions that have been used earlier. The result applies also…
The following problem is considered: given a joint distribution $P_{XY}$ and an event $E$, bound $P_{XY}(E)$ in terms of $P_XP_Y(E)$ (where $P_XP_Y$ is the product of the marginals of $P_{XY}$) and a measure of dependence of $X$ and $Y$.…
Our capacity to process information depends on the computational power at our disposal. Information theory captures our ability to distinguish states or communicate messages when it is unconstrained with unrivaled beauty and elegance. For…
We establish a quantitative bound on the entropy jump associated to the sum of independent, identically distributed (IID) radially symmetric random vectors having dimension greater than one. Following the usual approach, we first consider…
There is a vast body of recent literature on the reliability of communication through noisy channels, the recovery of community structures in the stochastic block model, the limiting behavior of the free entropy in spin glasses and the…
We derive upper bounds for random design linear regression with dependent ($\beta$-mixing) data absent any realizability assumptions. In contrast to the strictly realizable martingale noise regime, no sharp instance-optimal non-asymptotics…
Informational dependence between statistical or quantum subsystems can be described with Fisher matrix or Fubini-Study metric obtained from variations of the sample/configuration space coordinates. Using these non-covariant objects as…
Recently a new type of central limit theorem for belief functions was given in Epstein et al. [9]. In this paper, we generalize the central limit theorem in Epstein et al. [9] to accommodate general bounded random variables. These results…
In this paper we obtain a Bernstein type inequality for a class of weakly dependent and bounded random variables. The proofs lead to a moderate deviations principle for sums of bounded random variables with exponential decay of the strong…
We discuss sufficient conditions that guarantee the existence of asymptotic expansions for the Central Limit Theorem for weakly dependent random variables including observations arising from sufficiently chaotic dynamical systems like…
In this paper, under mild assumptions, we derive a law of large numbers, a central limit theorem with an error estimate, an almost sure invariance principle and a variant of Chernoff bound in finite-state hidden Markov models. These limit…
We present a new and simple approach to concentration inequalities for functions around their expectation with respect to non-product measures, i.e., for dependent random variables. Our method is based on coupling ideas and does not use…
We give general conditions for the central limit theorem and weak convergence to Brownian motion (the weak invariance principle / functional central limit theorem) to hold for observables of compact group extensions of nonuniformly…
Using proof-theoretic methods in the style of proof mining, we give novel computationally effective limit theorems for the convergence of the Cesaro-means of certain sequences of random variables. These results are intimately related to…
Information plays an important role in our understanding of the physical world. We hence propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical world,…
Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…
The Rotar central limit theorem is a remarkable theorem in the non-classical version since it does not use the condition of asymptotic infinitesimality for the independent individual summands, unlike the theorems named Lindeberg's and…
We introduce an inequality which may be viewed as a generalization of both the Brascamp-Lieb inequality and its reverse (Barthe's inequality), and prove its information-theoretic (i.e.\ entropic) formulation. This result leads to a unified…