Related papers: Relations between Information and Estimation in Di…
The paper develops new methods of non-parametric estimation a compound Poisson distribution. Such a problem arise, in particular, in the inference of a Levy process recorded at equidistant time intervals. Our key estimator is based on…
When estimating the directed information between two jointly stationary Markov processes, it is typically assumed that the recipient of the directed information is itself Markov of the same order as the joint process. While this assumption…
In this paper, we prove the existence of fundamental relations between information theory and estimation theory for network-coded flows. When the network is represented by a directed graph G=(V, E) and under the assumption of uncorrelated…
Optimization methods aimed at estimating the capacities of a general Gaussian channel are developed. Specifically evaluation of classical capacity as maximum of the Holevo information is pursued over all possible Gaussian encodings for the…
We prove an exact relationship between the optimal denoising function and the data distribution in the case of additive Gaussian noise, showing that denoising implicitly models the structure of data allowing it to be exploited in the…
This paper concerns the transmission of two independent Gaussian sources over a two-user decentralized interference channel, assuming that the transmitters are unaware of the instantaneous CSIs. The availability of the channel state…
In our companion paper [1], an information identity decomposition has been derived, which can be interpreted as a law of conservation of information flows in feedback systems. In this paper, we further investigate this decomposition result…
In this paper, we first provide three general norm inequalities, which are used to give new uncertainty relations of any finite observables and quantum channels via metric-adjusted skew information. The results are applicable to its special…
We study two-receiver Poisson channels using tools derived from stochastic calculus. We obtain a general formula for the mutual information over the Poisson channel that allows for conditioning and the use of auxiliary random variables. We…
We derive an approximate expression for mutual information in a broad class of discrete-time stationary channels with continuous input, under the constraint of vanishing input amplitude or power. The approximation describes the input by its…
We consider a linear Gaussian noise channel used with delayed feedback. The channel noise is assumed to be a ARMA (autoregressive and/or moving average) process. We reformulate the Gaussian noise channel into an intersymbol interference…
In deep learning, neural networks serve as noisy channels between input data and its representation. This perspective naturally relates deep learning with the pursuit of constructing channels with optimal performance in information…
We prove a new extremal inequality, motivated by the vector Gaussian broadcast channel and the distributed source coding with a single quadratic distortion constraint problems. As a corollary, this inequality yields a generalization of the…
We study data processing inequalities that are derived from a certain class of generalized information measures, where a series of convex functions and multiplicative likelihood ratios are nested alternately. While these information…
We consider the task of remote state estimation and stabilization of disturbed linear plants via noisy communication channels. In 2007 Matveev and Savkin established a surprising link between this problem and Shannon's theory of zero-error…
In this paper, we consider a class of symmetry groups associated to communication channels, which can informally be viewed as the transformations of the set of inputs that ``commute'' with the action of the channel. These groups were first…
The classical Binary Symmetric Channel has a fixed transition probability. We discuss the Binary Symmetric Channel with a variable transition probability that depends on a Poisson distribution. The error rate for this channel is determined…
Predictive statistical mechanics is a form of inference from available data, without additional assumptions, for predicting reproducible phenomena. By applying it to systems with Hamiltonian dynamics, a problem of predicting the macroscopic…
In some estimation problems, especially in applications dealing with information theory, signal processing and biology, theory provides us with additional information allowing us to restrict the parameter space to a finite number of points.…
An important objective of the classical processing of stationary random sequences under nonparametric uncertainty is the problem of filtering in case when the distribution of the underlying signal is unknown. In this paper it is assumed…