Related papers: Algorithm Families for Computing Information-Theor…
We consider the question of learning the natural parameters of a $k$ parameter minimal exponential family from i.i.d. samples in a computationally and statistically efficient manner. We focus on the setting where the support as well as the…
Universal source coding at short blocklengths is considered for an exponential family of distributions. The \emph{Type Size} code has previously been shown to be optimal up to the third-order rate for universal compression of all memoryless…
The framework of Integral Quadratic Constraints of Lessard et al. (2014) reduces the computation of upper bounds on the convergence rate of several optimization algorithms to semi-definite programming (SDP). Followup work by Nishihara et…
We revisit the high-dimensional content identification with lossy recovery problem (Tuncel and G\"und\"uz, 2014) and establish an exponential strong converse theorem. As a corollary of the exponential strong converse theorem, we derive an…
Discriminating between noisy quantum processes is a central primitive for quantum communication, metrology, and computing. While discrimination limits for finite-dimensional channels are well understood, the continuous-variable setting,…
We study the error exponents in quantum hypothesis testing between two sets of quantum states, extending the analysis beyond the independent and identically distributed case to encompass composite correlated hypotheses. In particular, we…
We study error bounds for linear programming decoding of regular LDPC codes. For memoryless binary-input output-symmetric channels, we prove bounds on the word error probability that are inverse doubly-exponential in the girth of the factor…
Structured prompts require integrating components according to task-relevant relations. How a network implements this integration is often hard to judge in language or vision, where those relations are rarely specified precisely enough to…
Consider the problem of detecting one of M i.i.d. Gaussian signals corrupted in white Gaussian noise. Conventionally, matched filters are used for detection. We first show that the outputs of the matched filter form a set of asymptotically…
For the information transmission over a binary symmetric channel the random coding is used. The transmission of exponential number of messages is considered. The exact decoding error probability exponent is derived. The proof is based on…
Coresets are efficient representations of data sets such that models trained on the coreset are provably competitive with models trained on the original data set. As such, they have been successfully used to scale up clustering models such…
Communicating information, like gradient vectors, between computing nodes in distributed and federated learning is typically an unavoidable burden, resulting in scalability issues. Indeed, communication might be slow and costly. Recent…
The analysis of random coding error exponents pertaining to erasure/list decoding, due to Forney, is revisited. Instead of using Jensen's inequality as well as some other inequalities in the derivation, we demonstrate that an exponentially…
Concatenated forward error correction is studied using an outer KP4 Reed-Solomon code with hard-decision decoding and inner single parity check (SPC) codes with Chase/Wagner soft-decision decoding. Analytical expressions are derived for the…
The ability to compare two degenerate probability distributions (i.e. two probability distributions supported on two distinct low-dimensional manifolds living in a much higher-dimensional space) is a crucial problem arising in the…
For the information transmission a binary symmetric channel is used. There is also another noisy binary symmetric channel (feedback channel), and the transmitter observes without delay all the outputs of the forward channel via that…
This work contains two main contributions concerning the asymmetric broadcast channel. The first is an analysis of the exact random coding error exponents for both users, and the second is the derivation of universal decoders for both…
We analyze the performance of a linear code used for a data compression of Slepian-Wolf type. In our framework, two correlated data are separately compressed into codewords employing Gallager-type codes and casted into a communication…
The partition function pertaining to finite--temperature decoding of a (typical) randomly chosen code is known to have three types of behavior, corresponding to three phases in the plane of rate vs. temperature: the {\it ferromagnetic…
This paper characterizes the optimal type-II error exponent for a distributed hypothesis testing-against-independence problem when the \emph{expected} rate of the sensor-detector link is constrained. Unlike for the well-known…