Related papers: Information Bottleneck on General Alphabets

We prove a strong Symmetry of Information relation for random strings (in the sense of Kolmogorov complexity) and establish tight bounds on the amount on nonuniformity that is necessary for extracting a string with randomness rate 1 from a…

Let $X_1, X_2, \ldots$ be a sequence of i.i.d. real-valued random variables with mean zero, and consider the scaled random walk of the form $Y^N_{k+1} = Y^N_{k} + a_N(Y^N_k) X_{k+1}$, where $a_N: \mathbb R \to \mathbb R_+$. We show, under…

When a variety of anomalous features motivate flagging different samples as outliers, Algorithmic Information Theory (AIT) offers a principled way to unify them in terms of a sample's randomness deficiency. Subject to the algorithmic Markov…

The problem of distributed data compression for function computation is considered, where (i) the function to be computed is not necessarily symbol-wise function and (ii) the information source has memory and may not be stationary nor…

Essentially all anytime-valid methods hinge on Ville's inequality to gain validity across time without incurring a union bound. Ville's inequality is a proper generalisation of Markov's inequality. It states that a non-negative…

In 1989, Thomassen asked whether there is an integer-valued function f(k) such that every f(k)-connected graph admits a spanning, bipartite $k$-connected subgraph. In this paper we take a first, humble approach, showing the conjecture is…

In this paper, we consider the information-theoretic characterization of the set of achievable rates and distortions in a broad class of multiterminal communication scenarios with general continuous-valued sources and channels. A framework…

Inferring the causal structure of a set of random variables from a finite sample of the joint distribution is an important problem in science. Recently, methods using additive noise models have been suggested to approach the case of…

Explaining deep learning models operating on time series data is crucial in various applications of interest which require interpretable and transparent insights from time series signals. In this work, we investigate this problem from an…

Let X=(X^0,\mu,T) be an ergodic measure preserving system. For a natural number k we consider the averages (*) 1/N \sum_{n=1}^N \prod_{j=1}^k f_j(T^{n a_j}x) where the functions f_j are bounded, and a_j are integers. A factor of X is…

We study a distributed learning problem in which Alice sends a compressed distillation of a set of training data to Bob, who uses the distilled version to best solve an associated learning problem. We formalize this as a rate-distortion…

Let $X^n$ be a uniformly distributed $n$-dimensional binary vector, and $Y^n$ be the result of passing $X^n$ through a binary symmetric channel (BSC) with crossover probability $\alpha$. A recent conjecture postulated by Courtade and Kumar…

We consider sequences of random variables whose probability generating functions are polynomials all of whose roots lie on the unit circle. The distribution of such random variables has only been sporadically studied in the literature. We…

Let f(n)=1 if n=1, 2^(2^(n-2)) if n \in {2,3,4,5}, (2+2^(2^(n-4)))^(2^(n-4)) if n \in {6,7,8,...}. We conjecture that if a system T \subseteq {x_i+1=x_k, x_i \cdot x_j=x_k: i,j,k \in {1,...,n}} has only finitely many solutions in positive…

Lossless variable-length source coding with unequal cost function is considered for general sources. In this problem, the codeword cost instead of codeword length is important. The infimum of average codeword cost has already been…

The information bottleneck (IB) method is a feasible defense solution against adversarial attacks in deep learning. However, this method suffers from the spurious correlation, which leads to the limitation of its further improvement of…

We study the randomized rumor spreading algorithm \emph{pull} on complete graphs with $n$ vertices. Starting with one informed vertex and proceeding in rounds, each vertex yet uninformed connects to a neighbor chosen uniformly at random and…

A short, information-theoretic proof of the Kac--Bernstein theorem, which is stated as follows, is presented: For any independent random variables $X$ and $Y$, if $X+Y$ and $X-Y$ are independent, then $X$ and $Y$ are normally distributed.

The Information Bottleneck (IB) is a method of lossy compression of relevant information. Its rate-distortion (RD) curve describes the fundamental tradeoff between input compression and the preservation of relevant information embedded in…

Folklore in complexity theory suspects that circuit lower bounds against $\mathbf{NC}^1$ or $\mathbf{P}/\operatorname{poly}$, currently out of reach, are a necessary step towards proving strong proof complexity lower bounds for systems like…