相关论文: Information Bottleneck on General Alphabets

Estimating individual level treatment effects (ITE) from observational data is a challenging and important area in causal machine learning and is commonly considered in diverse mission-critical applications. In this paper, we propose an…

The goal of this paper is to describe conditions which guarantee a central limit theorem for random variables, which distributions are controled by hidden Markov chains. We proved that when a Markov chain is ergodic and random variables…

We consider a scenario wherein two parties Alice and Bob are provided $X_{1}^{n}$ and $X_{2}^{n}$ -- samples that are IID from a PMF $P_{X_1 X_2}$. Alice and Bob can communicate to Charles over (noiseless) communication links of rate $R_1$…

Many algorithms are specified with respect to a fixed but unspecified parameter. Examples of this are especially common in cryptography, where protocols often feature a security parameter such as the bit length of a secret key. Our aim is…

We adapt arguments concerning entropy-theoretic convergence from the independent case to the case of FKG random variables. FKG systems are chosen since their dependence structure is controlled through covariance alone, though in the sequel…

We describe a proof of the Central Limit Theorem that has been formally verified in the Isabelle proof assistant. Our formalization builds upon and extends Isabelle's libraries for analysis and measure-theoretic probability. The proof of…

Random coding arguments are the backbone of most channel capacity achievability proofs. In this paper, we show that in their standard form, such arguments are insufficient for proving some network capacity theorems: structured coding…

The information bottleneck problem (IB) of jointly stationary Gaussian sources is considered. A water-filling solution for the IB rate is given in terms of its SNR spectrum and whose rate is attained via frequency domain test-channel…

We completely characterize the unimodal category for functions $f:\mathbb R\to[0,\infty)$ using a decomposition theorem obtained by generalizing the sweeping algorithm of Baryshnikov and Ghrist. We also give a characterization of the…

We obtain functional central limit theorems for both discrete time expressions of the form $1/\sqrt{N}\sum_{n=1}^{[Nt]}(F(X(q_1(n)),\ldots, X(q_{\ell}(n)))-\bar{F})$ and similar expressions in the continuous time where the sum is replaced…

We study the profile $X_{n,k}$ of random search trees including binary search trees and $m$-ary search trees. Our main result is a functional limit theorem of the normalized profile $X_{n,k}/\mathbb{E}X_{n,k}$ for $k=\lfloor\alpha\log…

Wyner's soft-covering lemma is a valuable tool for achievability proofs of information theoretic security, resolvability, channel synthesis, and source coding. The result herein sharpens the claim of soft-covering by moving away from an…

Learning invariant (causal) features for out-of-distribution (OOD) generalization has attracted extensive attention recently, and among the proposals invariant risk minimization (IRM) is a notable solution. In spite of its theoretical…

Let $\alpha$ be a Steinhaus or a Rademacher random multiplicative function. For a wide class of multiplicative functions $f$ we show that the sum $\sum_{n \le x}\alpha(n) f(n)$, normalised to have mean square $1$, has a non-Gaussian…

Bounds on the entropy of patterns of sequences generated by independently identically distributed (i.i.d.) sources are derived. A pattern is a sequence of indices that contains all consecutive integer indices in increasing order of first…

We study sums of a random multiplicative function; this is an example, of number-theoretic interest, of sums of products of independent random variables (chaoses). Using martingale methods, we establish a normal approximation for the sum…

The problem of joint universal source coding and modeling, addressed by Rissanen in the context of lossless codes, is generalized to fixed-rate lossy coding of continuous-alphabet memoryless sources. We show that, for bounded distortion…

Our understanding of information in systems has been based on the foundation of memoryless processes. Extensions to stable Markov and auto-regressive processes are classical. Berger proved a source coding theorem for the marginally unstable…

For N=1,2,..., let S_N be a simple random sample of size n=n_N from a population A_N of size N, where 0<=n<=N. Then with f_N=n/N, the sampling fraction, and 1_A the inclusion indicator that A is in S_N, for any H a subset of A_N of size k>=…

We prove the "Most informative boolean function" conjecture of Courtade and Kumar for high noise $\epsilon \ge 1/2 - \delta$, for some absolute constant $\delta > 0$. Namely, if $X$ is uniformly distributed in $\{0,1\}^n$ and $Y$ is…