Related papers: Information inequalities and a dependent Central L…
Fundamental limits on the controllability of physical systems are discussed in the light of information theory. It is shown that the second law of thermodynamics, when generalized to include information, sets absolute limits to the minimum…
Extremes of information combining inequalities play an important role in the analysis of sparse-graph codes under message-passing decoding. We introduce new tools for the derivation of such inequalities, and show by means of a concrete…
We study the number of occurrences of any fixed vincular permutation pattern. We show that this statistics on uniform random permutations is asymptotically normal and describe the speed of convergence. To prove this central limit theorem,…
We give simple proofs, under minimal hypotheses, of the Weak Law of Large Numbers and the Central Limit Theorem for independent identically distributed random variables. These proofs use only the elementary calculus, together with the most…
Shearer's inequality bounds the sum of joint entropies of random variables in terms of the total joint entropy. We give another lower bound for the same sum in terms of the individual entropies when the variables are functions of…
The central limit theorem is one of the most fundamental results in probability and has been successfully extended to locally dependent data and strongly-mixing random fields. In this paper, we establish its rate of convergence for…
We examine the minimization of information entropy for measures on the phase space of bounded domains, subject to constraints that are averages of grand canonical distributions. We describe the set of all such constraints and show that it…
This paper re-examines the limit theorems of Abadie and Imbens for nearest-neighbor matching estimators of average treatment effects with a fixed number of matches. We establish, for the first time, a non-normalized central limit theorem…
This dissertation investigates relative entropies, also called generalized divergences, and how they can be used to characterize information-theoretic tasks in quantum information theory. The main goal is to further refine characterizations…
A central limit theorem for arrays of symmetric row-wise exchangeable random variables is presented. The result is valid for finite and infinite extendable and non-extendable sequences. Unlike most reported versions of the central limit…
As conventional communication systems based on classic information theory have closely approached the limits of Shannon channel capacity, semantic communication has been recognized as a key enabling technology for the further improvement of…
We develop large sample theory for merged data from multiple sources. Main statistical issues treated in this paper are (1) the same unit potentially appears in multiple datasets from overlapping data sources, (2) duplicated items are not…
We consider the problem of bounding large deviations for non-i.i.d. random variables that are allowed to have arbitrary dependencies. Previous works typically assumed a specific dependence structure, namely the existence of independent…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
We study vectors chosen at random from a compact convex polytope in $\mathbb{R}^n$ given by a finite number of linear constraints. We determine which projections of these random vectors are asymptotically normal as $n\to\infty$. Marginal…
We prove limit theorems for sums of randomly chosen random variables conditioned on the summands. We consider several versions of the corner growth setting, including specific cases of dependence amongst the summands and summands with heavy…
Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a…
Weak convergence of maxima of dependent sequences of identically distributed continuous random variables is studied under normalizing sequences arising as subsequences of the normalizing sequences from an associated iid sequence. This…
Information theory provides ideas for conceptualising information and measuring relationships between objects. It has found wide application in the sciences, but economics and finance have made surprisingly little use of it. We show that…
In this paper, we obtain precise rates of convergence in the strong invariance principle for stationary sequences of real-valued random variables satisfying weak dependence conditions including strong mixing in the sense of Rosenblatt…