Related papers: Information Complexity and Estimation
The noisy-or and its generalization noisy-max have been utilized to reduce the complexity of knowledge acquisition. In this paper, we present a new representation of noisy-max that allows for efficient inference in general Bayesian…
Suppose that we wish to estimate a user's preference vector $w$ from paired comparisons of the form "does user $w$ prefer item $p$ or item $q$?," where both the user and items are embedded in a low-dimensional Euclidean space with distances…
The infinite-dimensional information operator for the nuisance parameter plays a key role in semiparametric inference, as it is closely related to the regular estimability of the target parameter. Calculation of information operators has…
We consider the problem of estimating a smooth functional of an unknown signal with discontinuity from Gaussian observations. The signal is a known function that depends on an unknown parameter. This problem is closely related to the famous…
We show that the global balance of information dynamics for general quantum measurements given in [F. Buscemi, M. Hayashi, and M. Horodecki, Phys.Rev.Lett. 100, 210504 (2008)] makes it possible to unify various and generally inequivalent…
This paper presents an unusual view of interference wireless networks based on complex system thinking. To proceed with this analysis, a literature review of the different applications of complex systems is firstly presented to illustrate…
A simple model for image formation in linear shift-invariant systems is considered, in which both the detected signal and the noise variance are varying slowly compared to the point-spread function of the system. It is shown that within the…
Information theory, though originally developed for communications engineering, provides mathematical tools with broad applications across science. These tools characterize the fundamental limits of data compression and transmission in the…
We revisit the long-standing question of the relation between image appreciation and its statistical properties. We generate two different sets of random images well distributed along three measures of entropic complexity. We run a…
The log-likelihood of a generative model often involves both positive and negative terms. For a temporal multivariate point process, the negative term sums over all the possible event types at each time and also integrates over all the…
The introduction of the partial information decomposition generated a flurry of proposals for defining an intersection information that quantifies how much of "the same information" two or more random variables specify about a target random…
We present a measurement noise reduction scheme based on information flow of a chaotic system. This scheme operates on conditions of chaoticity and well-defined noise level, not depending on other detailed characteristics of noise. Starting…
Capturing the interesting components of an image is a key aspect of image understanding. When a speaker annotates an image, selecting labels that are informative greatly depends on the prior knowledge of a prospective listener. Motivated by…
In this work, conditional entropy is used to quantify the information loss induced by passing a continuous random variable through a memoryless nonlinear input-output system. We derive an expression for the information loss depending on the…
The weighted entropy $H^{\rm w}_\phi (X)=H^{\rm w}_\phi (f)$ of a random variable $X$ with values $x$ and a probability-mass/density function $f$ is defined as the mean value ${\mathbb E} I^{\rm w}_\phi(X)$ of the weighted information…
In this paper, we study the remote estimation problem of a Markov process over a channel with a cost. We formulate this problem as an infinite horizon optimization problem with two players, i.e., a sensor and a monitor, that have distinct…
Given an original discrete source X with the distribution p_X that is corrupted by noise to produce the noisy data Y with the given joint distribution p(X, Y). A quantizer/classifier Q : Y -> Z is then used to classify/quantize the data Y…
Identifying meaningful signal buried in noise is a problem of interest arising in diverse scenarios of data-driven modeling. We present here a theoretical framework for exploiting intrinsic geometry in data that resists noise corruption,…
Efficient information processing is crucial for both living organisms and engineered systems. The mutual information rate, a core concept of information theory, quantifies the amount of information shared between the trajectories of input…
Mathematical models of real life phenomena are highly nonlinear involving multiple parameters and often exhibiting complex dynamics. Experimental data sets are typically small and noisy, rendering estimation of parameters from such data…