Related papers: Information Complexity and Estimation
In a topological dynamical system the complexity of an orbit is a measure of the amount of information (algorithmic information content) that is necessary to describe the orbit. This indicator is invariant up to topological conjugation. We…
Linear fusion of estimates under the condition of no knowledge of correlation of estimation errors has reached maturity. On the other hand, various cases of partial knowledge are still active research areas. A frequent motivation is to deal…
Inspired by Solomonoffs theory of inductive inference, we propose a prior based on circuit complexity. There are several advantages to this approach. First, it relies on a complexity measure that does not depend on the choice of UTM. There…
This paper addresses the problem of learning linear dynamical systems from noisy observations. In this setting, existing algorithms either yield biased parameter estimates or have large sample complexities. We resolve these issues by…
We explore a model of non-Bayesian information aggregation in networks. Agents non-cooperatively choose among Friedkin-Johnsen type aggregation rules to maximize payoffs. The DeGroot rule is chosen in equilibrium if and only if there is…
We proposed a learning algorithm for nonparametric estimation and on-line prediction for general stationary ergodic sources. We prepare histograms each of which estimates the probability as a finite distribution, and mixture them with…
A notion of directed information between two continuous-time processes is proposed. A key component in the definition is taking an infimum over all possible partitions of the time interval, which plays a role no less significant than the…
Common information (CI) is ubiquitous in information theory and related areas such as theoretical computer science and discrete probability. However, because there are multiple notions of CI, a unified understanding of the deep…
Decision making in modern stochastic systems, including e-commerce platforms, financial markets and healthcare systems, has evolved into a multifaceted process that combines information acquisition and adaptive information sources. This…
This paper investigates properties of noisy quantum information channels. We define a new quantity called {\em coherent information} which measures the amount of quantum information conveyed in the noisy channel. This quantity can never be…
In this paper we suggest a new algorithm for determination of signal-to-noise ratio (SNR). SNR is a quantitative measure widely used in science and engineering. Generally, methods for determination of SNR are based on using of…
In the simplest sequential decision problem for an ergodic stochastic process X, at each time n a decision u_n is made as a function of past observations X_0,...,X_{n-1}, and a loss l(u_n,X_n) is incurred. In this setting, it is known that…
We consider the problem of recovering linear image of unknown signal belonging to a given convex compact signal set from noisy observation of another linear image of the signal. We develop a simple generic efficiently computable nonlinear…
This paper presents an analysis of the concept of capacity for noisy com- putations, i.e. functions implemented by unreliable or random devices. An information theoretic model of noisy computation of a perfect function f (measurable…
We consider the problem of pointwise estimation of multi-dimensional signals $s$, from noisy observations $(y_\tau)$ on the regular grid $\bZd$. Our focus is on the adaptive estimation in the case when the signal can be well recovered using…
This paper considers a sequential estimation and sensor scheduling problem in the presence of multiple communication channels. As opposed to the classical remote estimation problem that involves one perfect (noiseless) channel and one…
The goal of ordinal embedding is to represent items as points in a low-dimensional Euclidean space given a set of constraints in the form of distance comparisons like "item $i$ is closer to item $j$ than item $k$". Ordinal constraints like…
We try to establish a unified information theoretic approach to learning and to explore some of its applications. First, we define {\em predictive information} as the mutual information between the past and the future of a time series,…
Weighted Updating generalizes Bayesian updating, allowing for biased beliefs by weighting the likelihood function and prior distribution with positive real exponents. I provide a rigorous foundation for the model by showing that…
Equation learning aims to infer differential equation models from data. While a number of studies have shown that differential equation models can be successfully identified when the data are sufficiently detailed and corrupted with…