Related papers: Measuring Information from Moments
An information theoretic measure is derived that quantifies the statistical coherence between systems evolving in time. The standard time delayed mutual information fails to distinguish information that is actually exchanged from shared…
This paper explores the effects of simulated moments on the performance of inference methods based on moment inequalities. Commonly used confidence sets for parameters are level sets of criterion functions whose boundary points may depend…
Objective estimators of multimedia quality are often judged by comparing estimates with subjective "truth data," most often via Pearson correlation coefficient (PCC) or mean-squared error (MSE). But subjective test results contain noise, so…
Given data drawn from a collection of Gaussian variables with a common mean but different and unknown variances, what is the best algorithm for estimating their common mean? We present an intuitive and efficient algorithm for this task. As…
Data from spectrophotometers form vectors of a large number of exploitable variables. Building quantitative models using these variables most often requires using a smaller set of variables than the initial one. Indeed, a too large number…
The problem of the optimal allocation (in the expected mean square error sense) of a measurement budget for particle filtering is addressed. We propose three different optimal intermittent filters, whose optimality criteria depend on the…
We consider generalized linear models in regimes where the number of nonzero components of the signal and accessible data points are sublinear with respect to the size of the signal. We prove a variational formula for the asymptotic mutual…
The representation of a given quantity with less information is often referred to as `quantization' and it is an important subject in information theory. In this paper, we have considered absolutely continuous probability measures on unit…
In this paper, an alternative approximation to the innovation method is introduced for the parameter estimation of diffusion processes from partial and noisy observations. This is based on a convergent approximation to the first two…
We present simple differentially private estimators for the mean and covariance of multivariate sub-Gaussian data that are accurate at small sample sizes. We demonstrate the effectiveness of our algorithms both theoretically and empirically…
Maximizing the precision in estimating parameters in a quantum system subject to instrumentation constraints is cast as a convex optimization problem. We account for prior knowledge about the parameter range by developing a worst-case and…
A notion of directed information between two continuous-time processes is proposed. A key component in the definition is taking an infimum over all possible partitions of the time interval, which plays a role no less significant than the…
To compare different forecasting methods on demand series we require an error measure. Many error measures have been proposed, but when demand is intermittent some become inapplicable, some give counter-intuitive results, and there is no…
In this paper we have considered the problem of estimating the population mean in systematic sampling using information on an auxiliary variable in presence of non response. Some modified ratio, product and difference type estimators in…
The problem of distributed estimation of a parametric physical field is stated as a maximum likelihood estimation problem. Sensor observations are distorted by additive white Gaussian noise. Prior to data transmission, each sensor quantizes…
A general information transmission model, under independent and identically distributed Gaussian codebook and nearest neighbor decoding rule with processed channel output, is investigated using the performance metric of generalized mutual…
Fundamental limitations or performance trade-offs/limits are important properties and constraints of both control and filtering systems. Among various trade-off metrics, total information rate that characterizes the sensitivity trade-offs…
In data-driven learning and inference tasks, the high cost of acquiring samples from the target distribution often limits performance. A common strategy to mitigate this challenge is to augment the limited target samples with data from a…
A framework for a quantum information theory is introduced that is based on the measure of quantum information associated with probability distribution predicted by quantum measuring of state. The entanglement between states of measured…
In this paper we focus on the estimation of mutual information from finite samples $(\mathcal{X}\times\mathcal{Y})$. The main concern with estimations of mutual information is their robustness under the class of transformations for which it…