Related papers: Measuring Information from Moments
In array processing, a common problem is to estimate the angles of arrival of $K$ deterministic sources impinging on an array of $M$ antennas, from $N$ observations of the source signal, corrupted by gaussian noise. The problem reduces to…
We use a novel form of quantum conditional probability to define new measures of quantum information in a dynamical context. We explore relationships between our new quantities and standard measures of quantum information, such as von…
Measurement models (MMs) stand at the highest structural level of quantum measurement theory. MMs can be employed to construct instruments which stand at the next level. An instrument is thought of as an apparatus that is used to measure…
We consider the processing of statistical samples $X\sim P_\theta$ by a channel $p(y|x)$, and characterize how the statistical information from the samples for estimating the parameter $\theta\in\mathbb{R}^d$ can scale with the mutual…
There is growing interest in improving our algorithmic understanding of fundamental statistical problems such as mean estimation, driven by the goal of understanding the limits of what we can extract from valuable data. The state of the art…
Based on information theory, we present a method to determine an optimal Markov approximation for modelling and prediction from time series data. The method finds a balance between minimal modelling errors by taking as much as possible…
We present two classes of improved estimators for mutual information $M(X,Y)$, from samples of random points distributed according to some joint probability density $\mu(x,y)$. In contrast to conventional estimators based on binnings, they…
In this article, we study the approximation of a probability measure $\mu$ on $\mathbb{R}^{d}$ by its empirical measure $\hat{\mu}_{N}$ interpreted as a random quantization. As error criterion we consider an averaged $p$-th moment…
We formulate meta learning using information theoretic concepts; namely, mutual information and the information bottleneck. The idea is to learn a stochastic representation or encoding of the task description, given by a training set, that…
Quantum sensors are among the most promising quantum technologies, allowing to attain the ultimate precision limit for parameter estimation. In order to achieve this, it is required to fully control and optimize what constitutes the…
We adopt and expand McDonald's (2011) regression framework for measurement precision, integrating two key perspectives: (a) reliability of observed scores and (b) optimal prediction of latent scores. Reliability arises from a measurement…
In Bayesian inference, we seek to compute information about random variables such as moments or quantiles on the basis of {available data} and prior information. When the distribution of random variables is {intractable}, Monte Carlo (MC)…
We introduce a new method for estimating the mean of an outcome variable within groups when researchers only observe the average of the outcome and group indicators across a set of aggregation units, such as geographical areas. Existing…
We derive lower bounds on the Bayes risk in decentralized estimation, where the estimator does not have direct access to the random samples generated conditionally on the random parameter of interest, but only to the data received from…
We address the problem of estimating the transmissivity of the pure-loss channel from the Bayesian point of view, i.e., we consider that some prior probability distribution function (PDF) on the unknown variable is available and we employ…
The problem of fast point-to-point MIMO channel mutual information estimation is addressed, in the situation where the receiver undergoes unknown colored interference, whereas the channel with the transmitter is perfectly known. The…
The effects of quantization and coding on the estimation quality of Gauss-Markov processes are considered, with a special attention to the Ornstein-Uhlenbeck process. Samples are acquired from the process, quantized, and then encoded for…
The problem how to approximately determine the absolute value of the Fisher information measure for a general parametric probabilistic system is considered. Having available the first and second moment of the system output in a parametric…
We consider the problem of sequentially learning to estimate, in the mean squared error (MSE) sense, a Gaussian $K$-vector of unknown covariance by observing only $m < K$ of its entries in each round. We propose two MSE estimators, and…
Calibration, the practice of choosing the parameters of a structural model to match certain empirical moments, can be viewed as minimum distance estimation. Existing standard error formulas for such estimators require a consistent estimate…