Related papers: Measuring Information from Moments
This paper considers pilot-based channel estimation in large-scale multiple-input multiple-output (MIMO) communication systems, also known as massive MIMO, where there are hundreds of antennas at one side of the link. Motivated by the fact…
Random sampling is an essential tool in the processing and transmission of data. It is used to summarize data too large to store or manipulate and meet resource constraints on bandwidth or battery power. Estimators that are applied to the…
This paper introduces measures for how each moment contributes to the precision of parameter estimates in GMM settings. For example, one of the measures asks what would happen to the variance of the parameter estimates if a particular…
We prove lower bounds on the error of any estimator for the mean of a real probability distribution under the knowledge that the distribution belongs to a given set. We apply these lower bounds both to parametric and nonparametric…
Information-theoretic quantities like entropy and mutual information have found numerous uses in machine learning. It is well known that there is a strong connection between these entropic quantities and submodularity since entropy over a…
Information theory is an outstanding framework to measure uncertainty, dependence and relevance in data and systems. It has several desirable properties for real world applications: it naturally deals with multivariate data, it can handle…
This paper considers estimation of a quantized constant in noise when using uniform and nonuniform quantizers. Estimators based on simple arithmetic averages, on sample statistical moments and on the maximum-likelihood procedure are…
The optimization of Value of Information (VoI) in sensor networks integrates awareness of the measured process in the communication system. However, most existing scheduling algorithms do not consider the specific needs of monitoring…
We consider the problem of subspace estimation in situations where the number of available snapshots and the observation dimension are comparable in magnitude. In this context, traditional subspace methods tend to fail because the…
This paper focuses on the minimum mean squared error (MMSE) channel estimator for multiple-input multiple-output (MIMO) systems with one-bit quantization at the receiver side. Despite its optimality and significance in estimation theory,…
An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Let $P_{S_n}$ be the distribution of a sum $S_n=\Sumn Y_i$ of…
Information diagram and the I-measure are useful mnemonics where random variables are treated as sets, and entropy and mutual information are treated as a signed measure. Although the I-measure has been successful in machine proofs of…
We propose two new measures for extracting the unique information in $X$ and not $Y$ about a message $M$, when $X, Y$ and $M$ are joint random variables with a given joint distribution. We take a Markov based approach, motivated by…
Informationally complete measurements allow the estimation of expectation values of any operator on a quantum system, by changing only the data-processing of the measurement outcomes. In particular, an informationally complete measurement…
This paper studies the effect of parametric mismatch in minimum mean square error (MMSE) estimation. In particular, we consider the problem of estimating the input signal from the output of an additive white Gaussian channel whose gain is…
The estimation of mutual information (MI) or conditional mutual information (CMI) from a set of samples is a long-standing problem. A recent line of work in this area has leveraged the approximation power of artificial neural networks and…
Quantum Metrology calculates the ultimate precision of all estimation strategies, measuring what is their root mean-square error (RMSE) and their Fisher information. Here, instead, we ask how many bits of the parameter we can recover,…
This paper focuses on a networked state estimation problem for a spatially large linear system with a distributed array of sensors, each of which offers partial state measurements, and the transmission is lossy. We propose a measurement…
This article expands the framework of Bayesian inference and provides direct probabilistic methods for approaching inference tasks that are typically handled with information theory. We treat Bayesian probability updating as a random…
Mutual Information is the metric that is used to perform link adaptation, which allows to achieve rates near capacity. The computation of adaptive transmission modes is achieved by employing the mapping between the Signal to Noise Ratio and…