Related papers: Measuring Information from Moments
The James-Stein estimator's dominance over maximum likelihood in terms of mean square error (MSE) has been one of the most celebrated results in modern statistics, suggesting that biased estimators can systematically outperform unbiased…
Detailed derivations of two bounds of the minimum mean-square error (MMSE) of complex-valued multiple-input multiple-output (MIMO) systems are proposed for performance evaluation. Particularly, the lower bound is derived based on a…
A measure to estimate the direct and directional coupling in multivariate time series is proposed. The measure is an extension of a recently published measure of conditional Mutual Information from Mixed Embedding (MIME) for bivariate time…
We propose an adversarial evaluation framework for sensitive feature inference based on minimum mean-squared error (MMSE) estimation with a finite sample size and linear predictive models. Our approach establishes theoretical lower bounds…
In recent years, a number of results have been developed which connect information measures and estimation measures under various models, including, predominently, Gaussian and Poisson models. More recent results due to Taborda and…
We introduce a novel Mutual Information (MI) estimator that fundamentally reframes the discriminative approach. Instead of training a classifier to discriminate between joint and marginal distributions, we learn a normalizing flow that…
In finite samples, the use of a slightly endogenous but highly relevant instrument can reduce mean-squared error (MSE). Building on this observation, I propose a novel moment selection procedure for GMM -- the Focused Moment Selection…
This paper addresses the development of analytical tools for the computation of the moments of random Gram matrices with one side correlation. Such a question is mainly driven by applications in signal processing and wireless communications…
Gaussian quantum probes have been widely used in quantum metrology and thermometry, where the goal is to estimate the temperature of an environment with which the probe interacts. It was recently shown that introducing initial…
Empirical Bayes estimators are based on minimizing the average risk with the hyper-parameters in the weighting function being estimated from observed data. The performance of an empirical Bayes estimator is typically evaluated by its mean…
An abstraction for multisensor communication termed the Gaussian Multiplex Channel is presented and analyzed. In this model, the sensor outputs can be added together in any combination through a network of switches, and the combinations can…
Estimation of mutual information between (multidimensional) real-valued variables is used in analysis of complex systems, biological systems, and recently also quantum systems. This estimation is a hard problem, and universally good…
Due to the absence of an external, classical time variable, the probabilistic predictions of covariant quantum theory are ambiguous when multiple measurements are considered. Here, we introduce an information theoretic framework to the…
The scalar additive Gaussian noise channel has the "single crossing point" property between the minimum-mean square error (MMSE) in the estimation of the input given the channel output, assuming a Gaussian input to the channel, and the MMSE…
Moment estimation is an important problem during circuit validation, in both pre-Silicon and post-Silicon stages. From the estimated moments, the probability of failure and parametric yield can be estimated at each circuit configuration and…
The term ``empirical predictor'' refers to a two-stage predictor of a linear combination of fixed and random effects. In the first stage, a predictor is obtained but it involves unknown parameters; thus, in the second stage, the unknown…
Probabilistic representation spaces convey information about a dataset and are shaped by factors such as the training data, network architecture, and loss function. Comparing the information content of such spaces is crucial for…
We incorporate into the empirical measure the auxiliary information given by a finite collection of expectation in an optimal information geometry way. This allows to unify several methods exploiting a side information and to uniquely…
This paper considers the fundamental limit of compressed sensing for i.i.d. signal distributions and i.i.d. Gaussian measurement matrices. Its main contribution is a rigorous characterization of the asymptotic mutual information (MI) and…
By employing various empirical estimators for the Mutual Information (MI) measure, we calculate and compare the estimates and their confidence intervals for both normal and non-normal bivariate data samples. We find that certain nonlinear…