Related papers: Information Complexity and Estimation
We introduce a signal processing model for signals in non-white noise, where the exact noise spectrum is a priori unknown. The model is based on a Student's t distribution and constitutes a natural generalization of the widely used normal…
Despite the popularity and success of deep learning, there is limited understanding of when, how, and why neural networks generalize to unseen examples. Since learning can be seen as extracting information from data, we formally study…
This paper considers a sequential sensor scheduling and remote estimation problem with one sensor and one estimator. The sensor makes sequential observations about the state of an underlying memoryless stochastic process and makes a…
Complex interactions between entities are often represented as edges in a network. In practice, the network is often constructed from noisy measurements and inevitably contains some errors. In this paper we consider the problem of…
Relative Fisher information, also known as score matching, is a recently introduced learning method for parameter estimation. Fundamental relations between relative entropy and score matching have been established in the literature for…
Complexity of patterns is a key information for human brain to differ objects of about the same size and shape. Like other innate human senses, the complexity perception cannot be easily quantified. We propose a transparent and universal…
Intrinsic noise in objective function and derivatives evaluations may cause premature termination of optimization algorithms. Evaluation complexity bounds taking this situation into account are presented in the framework of a deterministic…
We establish the necessary and sufficient conditions for unbiased estimation in multi-parameter estimation tasks. More specifically, we first consider quantum state estimation, where multiple parameters are encoded in a quantum state, and…
Noise is an important factor that influences the reliability of information acquisition, transmission, processing, and storage. In order to suppress the inevitable noise effects, a fault-tolerant information processing approach via quantum…
We propose a novel probabilistic method for detection of objects in noisy images. The method uses results from percolation and random graph theories. We present an algorithm that allows to detect objects of unknown shapes in the presence of…
In inverse problems, one attempts to infer spatially variable functions from indirect measurements of a system. To practitioners of inverse problems, the concept of "information" is familiar when discussing key questions such as which parts…
Extracting classical information from quantum systems is an essential step of many quantum algorithms. However, this information could be corrupted as the systems are prone to quantum noises, and its distortion under quantum dynamics has…
To account for the origin of information accumulation in nature despite the entropy-increase law, we advocate a universal mechanism due to competition/selection of general composite entities, from simple to complex. To confirm its…
The problem of distributed estimation of a parametric physical field is stated as a maximum likelihood estimation problem. Sensor observations are distorted by additive white Gaussian noise. Prior to data transmission, each sensor quantizes…
This paper considers the model of an arbitrary distributed signal x observed through an added independent white Gaussian noise w, y=x+w. New relations between the minimal mean square error of the non-causal estimator and the likelihood…
Fundamental relations between information and estimation have been established in the literature for the discrete-time Gaussian and Poisson channels. In this work, we demonstrate that such relations hold for a much larger class of…
Three variants of the statistical complexity function, which is used as a criterion in the problem of detection of a useful signal in the signal-noise mixture, are considered. The probability distributions maximizing the considered variants…
We study multivariate problems like function approximation, numerical integration, global optimization and dispersion. We obtain new results on the information complexity $n(\varepsilon,d)$ of these problems. The information complexity is…
We consider the response of a memoryless nonlinear device that converts an input signal $\xi(t)$ into an output $\eta(t)$ that only depends on the value of the input at the same time, $t$. For input Gaussian noise with power spectrum…
This paper studies the problem of finding the exact ranking from noisy comparisons. A comparison over a set of $m$ items produces a noisy outcome about the most preferred item, and reveals some information about the ranking. By repeatedly…