Related papers: Mutual Information and Optimality of Approximate M…
This paper considers a discrete-valued signal estimation scheme based on a low-complexity Bayesian optimal message passing algorithm (MPA) for solving massive linear inverse problems under highly correlated measurements. Gaussian belief…
In this chapter, using statistical physics methods, asymptotic closed-form expressions for the mean and variance of the mutual information for a multi-antenna transmitter-receiver pair in the presence of multiple Reconfigurable Intelligent…
We study distributed estimation of a Gaussian mean under communication constraints in a decision theoretical framework. Minimax rates of convergence, which characterize the tradeoff between the communication costs and statistical accuracy,…
Due to the massive number of devices in the M2M communication era, new challenges have been brought to the existing random-access (RA) mechanism, such as severe preamble collisions and resource block (RB) wastes. To address these problems,…
Multilayer (or deep) networks are powerful probabilistic models based on multiple stages of a linear transform followed by a non-linear (possibly random) function. In general, the linear transforms are defined by matrices and the non-linear…
We consider generalized linear models in regimes where the number of nonzero components of the signal and accessible data points are sublinear with respect to the size of the signal. We prove a variational formula for the asymptotic mutual…
Several strategies have been developed recently to ensure valid inference after model selection; some of these are easy to compute, while others fare better in terms of inferential power. In this paper, we consider a selective inference…
In this paper, we propose a distributed algorithm for solving loosely coupled problems with chordal sparsity which relies on primal-dual interior-point methods. We achieve this by distributing the computations at each iteration, using…
A simple model to study subspace clustering is the high-dimensional $k$-Gaussian mixture model where the cluster means are sparse vectors. Here we provide an exact asymptotic characterization of the statistically optimal reconstruction…
We study some of the most commonly used mutual information estimators, based on histograms of fixed or adaptive bin size, $k$-nearest neighbors and kernels, and focus on optimal selection of their free parameters. We examine the consistency…
Estimating the mutual information from samples from a joint distribution is a challenging problem in both science and engineering. In this work, we realize a variational bound that generalizes both discriminative and generative approaches.…
We consider the processing of statistical samples $X\sim P_\theta$ by a channel $p(y|x)$, and characterize how the statistical information from the samples for estimating the parameter $\theta\in\mathbb{R}^d$ can scale with the mutual…
Motivated by the recent interest in approximate message passing (AMP) for matrix-valued linear observations with superposition of \emph{multiple statistically asymmetric signal sources}, we introduce a multi-source AMP framework in which…
A distributed average consensus algorithm in which every sensor transmits with bounded peak power is proposed. In the presence of communication noise, it is shown that the nodes reach consensus asymptotically to a finite random variable…
Mutual information (MI) is a fundamental measure of statistical dependence between two variables, yet accurate estimation from finite data remains notoriously difficult. No estimator is universally reliable, and common approaches fail in…
In this paper, we investigate the additive Gaussian noise channel with noisy feedback. We consider the setup of linear coding of the feedback information and Gaussian signaling of the message (i.e. Cover-Pombra Scheme). Then, we derive the…
We consider the problem of signal estimation in generalized linear models defined via rotationally invariant design matrices. Since these matrices can have an arbitrary spectral distribution, this model is well suited for capturing complex…
This paper determines to within a single measurement the minimum number of measurements required to successfully reconstruct a signal drawn from a Gaussian mixture model in the low-noise regime. The method is to develop upper and lower…
In this work, we examine the optimality of Gaussian signalling for covert communications with an upper bound on $\mathcal{D}(p_{_1}||p_{_0})$ or $\mathcal{D}(p_{_0}||p_{_1})$ as the covertness constraint, where $\mathcal{D}(p_{_1}||p_{_0})$…
Consider the problem of estimating a low-rank matrix when its entries are perturbed by Gaussian noise. If the empirical distribution of the entries of the spikes is known, optimal estimators that exploit this knowledge can substantially…