Related papers: Information inequalities and a dependent Central L…
Let $\{X_n\}_{n\geq 1}$ be either a sequence of arbitrary random variables, or a martingale difference sequence, or a centered sequence with a suitable level of negative dependence. We prove Baum-Katz type theorems by only assuming that the…
The necessary information to distinguish a local inhomogeneous mass density field from its spatial average on a compact domain of the universe can be measured by relative information entropy. The Kullback-Leibler (KL) formula arises very…
We propose a novel approach to concentration for non-independent random variables. The main idea is to ``pretend'' that the random variables are independent and pay a multiplicative price measuring how far they are from actually being…
We give a characterization of Maximum Entropy/Minimum Relative Entropy inference by providing two `strong entropy concentration' theorems. These theorems unify and generalize Jaynes' `concentration phenomenon' and Van Campenhout and Cover's…
The data processing inequality is central to information theory and motivates the study of monotonic divergences. However, it is not clear operationally we need to consider all such divergences. We establish a simple method for Pinsker…
We derive a Gaussian Central Limit Theorem for the sample quantiles based on locally dependent random variables with explicit convergence rate. Our approach is based on converting the problem to a sum of indicator random variables, applying…
We prove Bernstein-type matrix concentration inequalities for linear combinations with matrix coefficients of binary random variables satisfying certain $\ell_\infty$-independence assumptions, complementing recent results by Kaufman, Kyng…
We obtain Central Limit Theorems in Functional form for a class of time-inhomogeneous interacting random walks on the simplex of probability measures over a finite set. Due to a reinforcement mechanism, the increments of the walks are…
The conditional entropy power inequality is a fundamental inequality in information theory, stating that the conditional entropy of the sum of two conditionally independent vector-valued random variables each with an assigned conditional…
We consider the analysis of probability distributions through their associated covariance operators from reproducing kernel Hilbert spaces. We show that the von Neumann entropy and relative entropy of these operators are intimately related…
Combining cross-section and time series data is a long and well established practice in empirical economics. We develop a central limit theory that explicitly accounts for possible dependence between the two data sets. We focus on common…
We generalize the Shannon's information theory in a nonadditive way by focusing on the source coding theorem. The nonadditive information content we adopted is consistent with the concept of the form invariance structure of the nonextensive…
Conditional mutual information is important in the selection and interpretation of graphical models. Its empirical version is well known as a generalised likelihood ratio test and that it may be represented as a difference in entropy. We…
We will show under very weak conditions on differentiability and dependence that the central limit theorem for quantiles holds and that the block bootstrap is weakly consistent. Under slightly stronger conditions, the bootstrap is strongly…
Concentration inequalities for the sample mean, like those due to Bernstein, Hoeffding, and Bentkus, are valid for any sample size but overly conservative, yielding confidence intervals that are unnecessarily wide. The central limit theorem…
We prove a central limit theorem with aassumptions which are many weak than classical conditions
We make use of an entropic property to establish a convergence theorem (Main Theorem), which reveals that the conditional entropy measures the asymptotic Gaussianity. As an application, we establish the {\it entropic conditional central…
For the power-law quantum wave packet in configuration space, the variance of the position observable may be divergent. Accordingly, the information-entropic formulation of the uncertainty principle becomes more appropriate than the…
When modelling metapopulation dynamics, the influence of a single patch on the metapopulation depends on the number of individuals in the patch. Since there is usually no obvious natural upper limit on the number of individuals in a patch,…
We study the mutual information estimation for mixed-pair random variables. One random variable is discrete and the other one is continuous. We develop a kernel method to estimate the mutual information between the two random variables. The…